org.apache.drill.exec.planner

Class ParquetPartitionDescriptor

java.lang.Object

org.apache.drill.exec.planner.AbstractPartitionDescriptor

org.apache.drill.exec.planner.ParquetPartitionDescriptor

All Implemented Interfaces:

Iterable<List<PartitionLocation>>, PartitionDescriptor

public class ParquetPartitionDescriptor
extends AbstractPartitionDescriptor

PartitionDescriptor that describes partitions based on column names instead of directory structure

Field Summary

Fields inherited from class org.apache.drill.exec.planner.AbstractPartitionDescriptor

locationSuperList, sublistsCreated

Fields inherited from interface org.apache.drill.exec.planner.PartitionDescriptor

PARTITION_BATCH_SIZE

Constructor Summary

Constructors

Constructor and Description
ParquetPartitionDescriptor(PlannerSettings settings, DrillScanRel scanRel)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	createPartitionSublists() Create sublists of the partition locations, each sublist of size at most PartitionDescriptor.PARTITION_BATCH_SIZE
org.apache.calcite.rel.core.TableScan	createTableScan(List<PartitionLocation> newPartitionLocation, boolean wasAllPartitionsPruned) Methods create a new TableScan rel node, given the lists of new partitions or new files to SCAN.
org.apache.calcite.rel.core.TableScan	createTableScan(List<PartitionLocation> newPartitionLocation, String cacheFileRoot, boolean wasAllPartitionsPruned, MetadataContext metaContext) Create a new TableScan rel node, given the lists of new partitions or new files to scan and a path to a metadata cache file
String	getBaseTableLocation()
Integer	getIdIfValid(String name) Check to see if the name is a partition name.
int	getMaxHierarchyLevel()
int	getPartitionHierarchyIndex(String partitionName)
TypeProtos.MajorType	getVectorType(SchemaPath column, PlannerSettings plannerSettings) Method returns the Major type associated with the given column
boolean	isPartitionName(String name)
void	populatePartitionVectors(ValueVector[] vectors, List<PartitionLocation> partitions, BitSet partitionColumnBitSet, Map<Integer,String> fieldNameMap) Method creates an in memory representation of all the partitions.

Methods inherited from class org.apache.drill.exec.planner.AbstractPartitionDescriptor

iterator, supportsMetadataCachePruning

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetPartitionDescriptor

public ParquetPartitionDescriptor(PlannerSettings settings,
                          DrillScanRel scanRel)

Method Detail

getPartitionHierarchyIndex

public int getPartitionHierarchyIndex(String partitionName)

isPartitionName

public boolean isPartitionName(String name)

getIdIfValid

public Integer getIdIfValid(String name)

Description copied from interface: PartitionDescriptor

Check to see if the name is a partition name.

Parameters:

name - The field name you want to compare to partition names.

Returns:

Return index if valid, otherwise return null;

getMaxHierarchyLevel

public int getMaxHierarchyLevel()

populatePartitionVectors

public void populatePartitionVectors(ValueVector[] vectors,
                            List<PartitionLocation> partitions,
                            BitSet partitionColumnBitSet,
                            Map<Integer,String> fieldNameMap)

Description copied from interface: PartitionDescriptor

Method creates an in memory representation of all the partitions. For each level of partitioning we will create a value vector which this method will populate for all the partitions with the values of the partitioning key

Parameters:

vectors - - Array of vectors in the container that need to be populated

partitions - - List of all the partitions that exist in the table

partitionColumnBitSet - - Partition columns selected in the query

fieldNameMap - - Maps field ordinal to the field name

getVectorType

public TypeProtos.MajorType getVectorType(SchemaPath column,
                                 PlannerSettings plannerSettings)

Description copied from interface: PartitionDescriptor

Method returns the Major type associated with the given column

Parameters:

column - - column whose type should be determined

Returns:

getBaseTableLocation

public String getBaseTableLocation()

createPartitionSublists

protected void createPartitionSublists()

Description copied from class: AbstractPartitionDescriptor

Create sublists of the partition locations, each sublist of size at most PartitionDescriptor.PARTITION_BATCH_SIZE

Specified by:

createPartitionSublists in class AbstractPartitionDescriptor

createTableScan

public org.apache.calcite.rel.core.TableScan createTableScan(List<PartitionLocation> newPartitionLocation,
                                                    String cacheFileRoot,
                                                    boolean wasAllPartitionsPruned,
                                                    MetadataContext metaContext)
                                                      throws Exception

Description copied from interface: PartitionDescriptor

Create a new TableScan rel node, given the lists of new partitions or new files to scan and a path to a metadata cache file

Specified by:

createTableScan in interface PartitionDescriptor

Overrides:

createTableScan in class AbstractPartitionDescriptor

Returns:

Throws:

Exception

createTableScan

public org.apache.calcite.rel.core.TableScan createTableScan(List<PartitionLocation> newPartitionLocation,
                                                    boolean wasAllPartitionsPruned)
                                                      throws Exception

Description copied from interface: PartitionDescriptor

Methods create a new TableScan rel node, given the lists of new partitions or new files to SCAN.

Returns:

Throws:

Exception