org.apache.drill.exec.planner

Class FileSystemPartitionDescriptor

java.lang.Object

org.apache.drill.exec.planner.AbstractPartitionDescriptor

org.apache.drill.exec.planner.FileSystemPartitionDescriptor

All Implemented Interfaces:

Iterable<List<PartitionLocation>>, PartitionDescriptor

public class FileSystemPartitionDescriptor
extends AbstractPartitionDescriptor

Field Summary

Fields inherited from class org.apache.drill.exec.planner.AbstractPartitionDescriptor

locationSuperList, sublistsCreated

Fields inherited from interface org.apache.drill.exec.planner.PartitionDescriptor

PARTITION_BATCH_SIZE

Constructor Summary

Constructors

Constructor and Description
FileSystemPartitionDescriptor(PlannerSettings settings, org.apache.calcite.rel.core.TableScan scanRel)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	createPartitionSublists() Create sublists of the partition locations, each sublist of size at most PartitionDescriptor.PARTITION_BATCH_SIZE
org.apache.calcite.rel.core.TableScan	createTableScan(List<PartitionLocation> newPartitionLocation, boolean wasAllPartitionsPruned) Methods create a new TableScan rel node, given the lists of new partitions or new files to SCAN.
org.apache.calcite.rel.core.TableScan	createTableScan(List<PartitionLocation> newPartitionLocation, String cacheFileRoot, boolean wasAllPartitionsPruned, MetadataContext metaContext) Create a new TableScan rel node, given the lists of new partitions or new files to scan and a path to a metadata cache file
String	getBaseTableLocation()
protected org.apache.calcite.util.Pair<Collection<String>,Boolean>	getFileLocationsAndStatus()
Integer	getIdIfValid(String name) Check to see if the name is a partition name.
int	getMaxHierarchyLevel()
String	getName(int index)
int	getPartitionHierarchyIndex(String partitionName)
DrillTable	getTable()
TypeProtos.MajorType	getVectorType(SchemaPath column, PlannerSettings plannerSettings) Method returns the Major type associated with the given column
boolean	isPartitionName(String name)
void	populatePartitionVectors(ValueVector[] vectors, List<PartitionLocation> partitions, BitSet partitionColumnBitSet, Map<Integer,String> fieldNameMap) Method creates an in memory representation of all the partitions.
boolean	supportsMetadataCachePruning()

Methods inherited from class org.apache.drill.exec.planner.AbstractPartitionDescriptor

iterator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileSystemPartitionDescriptor

public FileSystemPartitionDescriptor(PlannerSettings settings,
                             org.apache.calcite.rel.core.TableScan scanRel)

Method Detail

getPartitionHierarchyIndex

public int getPartitionHierarchyIndex(String partitionName)

isPartitionName

public boolean isPartitionName(String name)

getIdIfValid

public Integer getIdIfValid(String name)

Description copied from interface: PartitionDescriptor

Check to see if the name is a partition name.

Parameters:

name - The field name you want to compare to partition names.

Returns:

Return index if valid, otherwise return null;

getMaxHierarchyLevel

public int getMaxHierarchyLevel()

getTable

public DrillTable getTable()

populatePartitionVectors

public void populatePartitionVectors(ValueVector[] vectors,
                            List<PartitionLocation> partitions,
                            BitSet partitionColumnBitSet,
                            Map<Integer,String> fieldNameMap)

Description copied from interface: PartitionDescriptor

Method creates an in memory representation of all the partitions. For each level of partitioning we will create a value vector which this method will populate for all the partitions with the values of the partitioning key

Parameters:

vectors - - Array of vectors in the container that need to be populated

partitions - - List of all the partitions that exist in the table

partitionColumnBitSet - - Partition columns selected in the query

fieldNameMap - - Maps field ordinal to the field name

getVectorType

public TypeProtos.MajorType getVectorType(SchemaPath column,
                                 PlannerSettings plannerSettings)

Description copied from interface: PartitionDescriptor

Method returns the Major type associated with the given column

Parameters:

column - - column whose type should be determined

Returns:

getName

public String getName(int index)

getBaseTableLocation

public String getBaseTableLocation()

createPartitionSublists

protected void createPartitionSublists()

Description copied from class: AbstractPartitionDescriptor

Create sublists of the partition locations, each sublist of size at most PartitionDescriptor.PARTITION_BATCH_SIZE

Specified by:

createPartitionSublists in class AbstractPartitionDescriptor

getFileLocationsAndStatus

protected org.apache.calcite.util.Pair<Collection<String>,Boolean> getFileLocationsAndStatus()

createTableScan

public org.apache.calcite.rel.core.TableScan createTableScan(List<PartitionLocation> newPartitionLocation,
                                                    String cacheFileRoot,
                                                    boolean wasAllPartitionsPruned,
                                                    MetadataContext metaContext)
                                                      throws Exception

Description copied from interface: PartitionDescriptor

Create a new TableScan rel node, given the lists of new partitions or new files to scan and a path to a metadata cache file

Specified by:

createTableScan in interface PartitionDescriptor

Overrides:

createTableScan in class AbstractPartitionDescriptor

Returns:

Throws:

Exception

createTableScan

public org.apache.calcite.rel.core.TableScan createTableScan(List<PartitionLocation> newPartitionLocation,
                                                    boolean wasAllPartitionsPruned)
                                                      throws Exception

Description copied from interface: PartitionDescriptor

Methods create a new TableScan rel node, given the lists of new partitions or new files to SCAN.

Returns:

Throws:

Exception

supportsMetadataCachePruning

public boolean supportsMetadataCachePruning()

Specified by:

supportsMetadataCachePruning in interface PartitionDescriptor

Overrides:

supportsMetadataCachePruning in class AbstractPartitionDescriptor