org.apache.drill.exec.store.parquet

Class ParquetRecordWriter

java.lang.Object

org.apache.drill.exec.store.AbstractRecordWriter

org.apache.drill.exec.store.ParquetOutputRecordWriter

org.apache.drill.exec.store.parquet.ParquetRecordWriter

All Implemented Interfaces:

RecordWriter

public class ParquetRecordWriter
extends ParquetOutputRecordWriter

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ParquetRecordWriter.MapParquetConverter
class	ParquetRecordWriter.RepeatedMapParquetConverter

Nested classes/interfaces inherited from class org.apache.drill.exec.store.ParquetOutputRecordWriter

ParquetOutputRecordWriter.BigIntParquetConverter, ParquetOutputRecordWriter.BitParquetConverter, ParquetOutputRecordWriter.DateParquetConverter, ParquetOutputRecordWriter.Decimal18ParquetConverter, ParquetOutputRecordWriter.Decimal28DenseParquetConverter, ParquetOutputRecordWriter.Decimal28SparseParquetConverter, ParquetOutputRecordWriter.Decimal38DenseParquetConverter, ParquetOutputRecordWriter.Decimal38SparseParquetConverter, ParquetOutputRecordWriter.Decimal9ParquetConverter, ParquetOutputRecordWriter.Float4ParquetConverter, ParquetOutputRecordWriter.Float8ParquetConverter, ParquetOutputRecordWriter.IntervalDayParquetConverter, ParquetOutputRecordWriter.IntervalParquetConverter, ParquetOutputRecordWriter.IntervalYearParquetConverter, ParquetOutputRecordWriter.IntParquetConverter, ParquetOutputRecordWriter.NullableBigIntParquetConverter, ParquetOutputRecordWriter.NullableBitParquetConverter, ParquetOutputRecordWriter.NullableDateParquetConverter, ParquetOutputRecordWriter.NullableDecimal18ParquetConverter, ParquetOutputRecordWriter.NullableDecimal28DenseParquetConverter, ParquetOutputRecordWriter.NullableDecimal28SparseParquetConverter, ParquetOutputRecordWriter.NullableDecimal38DenseParquetConverter, ParquetOutputRecordWriter.NullableDecimal38SparseParquetConverter, ParquetOutputRecordWriter.NullableDecimal9ParquetConverter, ParquetOutputRecordWriter.NullableFloat4ParquetConverter, ParquetOutputRecordWriter.NullableFloat8ParquetConverter, ParquetOutputRecordWriter.NullableIntervalDayParquetConverter, ParquetOutputRecordWriter.NullableIntervalParquetConverter, ParquetOutputRecordWriter.NullableIntervalYearParquetConverter, ParquetOutputRecordWriter.NullableIntParquetConverter, ParquetOutputRecordWriter.NullableSmallIntParquetConverter, ParquetOutputRecordWriter.NullableTimeParquetConverter, ParquetOutputRecordWriter.NullableTimeStampParquetConverter, ParquetOutputRecordWriter.NullableTinyIntParquetConverter, ParquetOutputRecordWriter.NullableUInt1ParquetConverter, ParquetOutputRecordWriter.NullableUInt2ParquetConverter, ParquetOutputRecordWriter.NullableUInt4ParquetConverter, ParquetOutputRecordWriter.NullableUInt8ParquetConverter, ParquetOutputRecordWriter.NullableVar16CharParquetConverter, ParquetOutputRecordWriter.NullableVarBinaryParquetConverter, ParquetOutputRecordWriter.NullableVarCharParquetConverter, ParquetOutputRecordWriter.RepeatedBigIntParquetConverter, ParquetOutputRecordWriter.RepeatedBitParquetConverter, ParquetOutputRecordWriter.RepeatedDateParquetConverter, ParquetOutputRecordWriter.RepeatedDecimal18ParquetConverter, ParquetOutputRecordWriter.RepeatedDecimal28DenseParquetConverter, ParquetOutputRecordWriter.RepeatedDecimal28SparseParquetConverter, ParquetOutputRecordWriter.RepeatedDecimal38DenseParquetConverter, ParquetOutputRecordWriter.RepeatedDecimal38SparseParquetConverter, ParquetOutputRecordWriter.RepeatedDecimal9ParquetConverter, ParquetOutputRecordWriter.RepeatedFloat4ParquetConverter, ParquetOutputRecordWriter.RepeatedFloat8ParquetConverter, ParquetOutputRecordWriter.RepeatedIntervalDayParquetConverter, ParquetOutputRecordWriter.RepeatedIntervalParquetConverter, ParquetOutputRecordWriter.RepeatedIntervalYearParquetConverter, ParquetOutputRecordWriter.RepeatedIntParquetConverter, ParquetOutputRecordWriter.RepeatedSmallIntParquetConverter, ParquetOutputRecordWriter.RepeatedTimeParquetConverter, ParquetOutputRecordWriter.RepeatedTimeStampParquetConverter, ParquetOutputRecordWriter.RepeatedTinyIntParquetConverter, ParquetOutputRecordWriter.RepeatedUInt1ParquetConverter, ParquetOutputRecordWriter.RepeatedUInt2ParquetConverter, ParquetOutputRecordWriter.RepeatedUInt4ParquetConverter, ParquetOutputRecordWriter.RepeatedUInt8ParquetConverter, ParquetOutputRecordWriter.RepeatedVar16CharParquetConverter, ParquetOutputRecordWriter.RepeatedVarBinaryParquetConverter, ParquetOutputRecordWriter.RepeatedVarCharParquetConverter, ParquetOutputRecordWriter.SmallIntParquetConverter, ParquetOutputRecordWriter.TimeParquetConverter, ParquetOutputRecordWriter.TimeStampParquetConverter, ParquetOutputRecordWriter.TinyIntParquetConverter, ParquetOutputRecordWriter.UInt1ParquetConverter, ParquetOutputRecordWriter.UInt2ParquetConverter, ParquetOutputRecordWriter.UInt4ParquetConverter, ParquetOutputRecordWriter.UInt8ParquetConverter, ParquetOutputRecordWriter.Var16CharParquetConverter, ParquetOutputRecordWriter.VarBinaryParquetConverter, ParquetOutputRecordWriter.VarCharParquetConverter

Field Summary

Fields

Modifier and Type	Field and Description
static String	DRILL_VERSION_PROPERTY
static String	WRITER_VERSION_PROPERTY

Constructor Summary

Constructors

Constructor and Description
ParquetRecordWriter(FragmentContext context, ParquetWriter writer)

Method Summary

Methods

Modifier and Type	Method and Description
void	abort()
void	checkForNewPartition(int index) Check if the writer should start a new partition, and if so, start a new partition
void	cleanup()
void	endRecord() Called after adding all fields in a particular record are added using add{TypeHolder}(fieldId, TypeHolder) methods.
EventBasedRecordWriter.FieldConverter	getNewMapConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewRepeatedMapConverter(int fieldId, String fieldName, FieldReader reader)
void	init(Map<String,String> writerOptions) Initialize the writer.
void	startRecord() Called before starting writing fields in a record.
void	updateSchema(VectorAccessible batch) Update the schema in RecordWriter.

Methods inherited from class org.apache.drill.exec.store.ParquetOutputRecordWriter

getNewBigIntConverter, getNewBitConverter, getNewDateConverter, getNewDecimal18Converter, getNewDecimal28DenseConverter, getNewDecimal28SparseConverter, getNewDecimal38DenseConverter, getNewDecimal38SparseConverter, getNewDecimal9Converter, getNewFloat4Converter, getNewFloat8Converter, getNewIntConverter, getNewIntervalConverter, getNewIntervalDayConverter, getNewIntervalYearConverter, getNewNullableBigIntConverter, getNewNullableBitConverter, getNewNullableDateConverter, getNewNullableDecimal18Converter, getNewNullableDecimal28DenseConverter, getNewNullableDecimal28SparseConverter, getNewNullableDecimal38DenseConverter, getNewNullableDecimal38SparseConverter, getNewNullableDecimal9Converter, getNewNullableFloat4Converter, getNewNullableFloat8Converter, getNewNullableIntConverter, getNewNullableIntervalConverter, getNewNullableIntervalDayConverter, getNewNullableIntervalYearConverter, getNewNullableSmallIntConverter, getNewNullableTimeConverter, getNewNullableTimeStampConverter, getNewNullableTinyIntConverter, getNewNullableUInt1Converter, getNewNullableUInt2Converter, getNewNullableUInt4Converter, getNewNullableUInt8Converter, getNewNullableVar16CharConverter, getNewNullableVarBinaryConverter, getNewNullableVarCharConverter, getNewRepeatedBigIntConverter, getNewRepeatedBitConverter, getNewRepeatedDateConverter, getNewRepeatedDecimal18Converter, getNewRepeatedDecimal28DenseConverter, getNewRepeatedDecimal28SparseConverter, getNewRepeatedDecimal38DenseConverter, getNewRepeatedDecimal38SparseConverter, getNewRepeatedDecimal9Converter, getNewRepeatedFloat4Converter, getNewRepeatedFloat8Converter, getNewRepeatedIntConverter, getNewRepeatedIntervalConverter, getNewRepeatedIntervalDayConverter, getNewRepeatedIntervalYearConverter, getNewRepeatedSmallIntConverter, getNewRepeatedTimeConverter, getNewRepeatedTimeStampConverter, getNewRepeatedTinyIntConverter, getNewRepeatedUInt1Converter, getNewRepeatedUInt2Converter, getNewRepeatedUInt4Converter, getNewRepeatedUInt8Converter, getNewRepeatedVar16CharConverter, getNewRepeatedVarBinaryConverter, getNewRepeatedVarCharConverter, getNewSmallIntConverter, getNewTimeConverter, getNewTimeStampConverter, getNewTinyIntConverter, getNewUInt1Converter, getNewUInt2Converter, getNewUInt4Converter, getNewUInt8Converter, getNewVar16CharConverter, getNewVarBinaryConverter, getNewVarCharConverter, setUp

Methods inherited from class org.apache.drill.exec.store.AbstractRecordWriter

getNewRepeatedListConverter, getNewUnionConverter, newPartition, setPartitionVector

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.store.RecordWriter

getNewRepeatedListConverter, getNewUnionConverter

Field Detail

DRILL_VERSION_PROPERTY

public static final String DRILL_VERSION_PROPERTY

See Also:

Constant Field Values

WRITER_VERSION_PROPERTY

public static final String WRITER_VERSION_PROPERTY

See Also:

Constant Field Values

Constructor Detail

ParquetRecordWriter

public ParquetRecordWriter(FragmentContext context,
                   ParquetWriter writer)
                    throws OutOfMemoryException

Throws:

OutOfMemoryException

Method Detail

init

public void init(Map<String,String> writerOptions)
          throws IOException

Description copied from interface: RecordWriter

Initialize the writer.

Parameters:

writerOptions - Contains key, value pair of settings.

Throws:

IOException

updateSchema

public void updateSchema(VectorAccessible batch)
                  throws IOException

Description copied from interface: RecordWriter

Update the schema in RecordWriter. Called at least once before starting writing the records.

Throws:

IOException

checkForNewPartition

public void checkForNewPartition(int index)

Description copied from interface: RecordWriter

Check if the writer should start a new partition, and if so, start a new partition

Specified by:

checkForNewPartition in interface RecordWriter

Overrides:

checkForNewPartition in class AbstractRecordWriter

getNewMapConverter

public EventBasedRecordWriter.FieldConverter getNewMapConverter(int fieldId,
                                                       String fieldName,
                                                       FieldReader reader)

Description copied from interface: RecordWriter

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewMapConverter in interface RecordWriter

Overrides:

getNewMapConverter in class AbstractRecordWriter

getNewRepeatedMapConverter

public EventBasedRecordWriter.FieldConverter getNewRepeatedMapConverter(int fieldId,
                                                               String fieldName,
                                                               FieldReader reader)

Specified by:

getNewRepeatedMapConverter in interface RecordWriter

Overrides:

getNewRepeatedMapConverter in class AbstractRecordWriter

startRecord

public void startRecord()
                 throws IOException

Description copied from interface: RecordWriter

Called before starting writing fields in a record.

Throws:

IOException

endRecord

public void endRecord()
               throws IOException

Description copied from interface: RecordWriter

Called after adding all fields in a particular record are added using add{TypeHolder}(fieldId, TypeHolder) methods.

Throws:

IOException

abort

public void abort()
           throws IOException

Throws:

IOException

cleanup

public void cleanup()
             throws IOException

Throws:

IOException