org.apache.hadoop.hive.metastore

Class PartitionIterable

java.lang.Object

org.apache.hadoop.hive.metastore.PartitionIterable

All Implemented Interfaces:

Iterable<Partition>

public class PartitionIterable
extends Object
implements Iterable<Partition>

PartitionIterable - effectively a lazy Iterable<Partition> Sometimes, we have a need for iterating through a list of partitions, but the list of partitions can be too big to fetch as a single object. Thus, the goal of PartitionIterable is to act as an Iterable<Partition> while lazily fetching each relevant partition, one after the other as independent metadata calls. It is very likely that any calls to PartitionIterable are going to result in a large number of calls, so use sparingly only when the memory cost of fetching all the partitions in one shot is too prohibitive. This is still pretty costly in that it would retain a list of partition names, but that should be far less expensive than the entire partition objects. Note that remove() is an illegal call on this, and will result in an IllegalStateException.

Constructor Summary

Constructors

Constructor and Description
PartitionIterable(Collection<Partition> ptnsProvided) Dummy constructor, which simply acts as an iterator on an already-present list of partitions, allows for easy drop-in replacement for other methods that already have a List<Partition>
PartitionIterable(IMetaStoreClient msc, Table table, int batch_size) Primary constructor that fetches all partitions in a given table, given a Hive object and a table object, and a partial partition spec.

Method Summary

Modifier and Type	Method and Description
List<String>	getPartitionNames(IMetaStoreClient msc, String catName, String dbName, String tblName, short max)
Iterator<Partition>	iterator()
PartitionIterable	withProjectSpec(GetPartitionsRequest request)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

PartitionIterable

public PartitionIterable(Collection<Partition> ptnsProvided)

Dummy constructor, which simply acts as an iterator on an already-present list of partitions, allows for easy drop-in replacement for other methods that already have a List<Partition>

PartitionIterable

public PartitionIterable(IMetaStoreClient msc,
                         Table table,
                         int batch_size)
                  throws MetastoreException

Primary constructor that fetches all partitions in a given table, given a Hive object and a table object, and a partial partition spec.

Throws:

MetastoreException

Method Detail

iterator

public Iterator<Partition> iterator()

Specified by:

iterator in interface Iterable<Partition>

getPartitionNames

public List<String> getPartitionNames(IMetaStoreClient msc,
                                      String catName,
                                      String dbName,
                                      String tblName,
                                      short max)
                               throws MetastoreException

Throws:

MetastoreException

withProjectSpec

public PartitionIterable withProjectSpec(GetPartitionsRequest request)