org.apache.hudi.common.table.log

Interface HoodieLogFormat

public interface HoodieLogFormat

File Format for Hoodie Log Files. The File Format consists of blocks each separated with a MAGIC sync marker. A Block can either be a Data block, Command block or Delete Block. Data Block - Contains log records serialized as Avro Binary Format Command Block - Specific commands like ROLLBACK_PREVIOUS-BLOCK - Tombstone for the previously written block Delete Block - List of keys to delete - tombstone for keys

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	HoodieLogFormat.LogFormatVersion A set of feature flags associated with a log format.
static interface	HoodieLogFormat.Reader Reader interface which is an Iterator of HoodieLogBlock.
static interface	HoodieLogFormat.Writer Writer interface to allow appending block to this file format.
static class	HoodieLogFormat.WriterBuilder Builder class to construct the default log format writer.

Field Summary

Fields

Modifier and Type	Field and Description
static int	CURRENT_VERSION The current version of the log format.
static String	DEFAULT_LOG_FORMAT_WRITER
static String	DEFAULT_WRITE_TOKEN
static byte[]	MAGIC Magic 6 bytes we put at the start of every block in the log file.
static String	UNKNOWN_WRITE_TOKEN

Method Summary

Modifier and Type	Method and Description
static HoodieLogFormat.Reader	newReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema)
static HoodieLogFormat.Reader	newReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, boolean reverseReader)
static HoodieLogFormat.WriterBuilder	newWriterBuilder()

Field Detail

MAGIC

static final byte[] MAGIC

Magic 6 bytes we put at the start of every block in the log file.

CURRENT_VERSION

static final int CURRENT_VERSION

The current version of the log format. Anytime the log format changes this version needs to be bumped and corresponding changes need to be made to HoodieLogFormatVersion

See Also:

Constant Field Values

UNKNOWN_WRITE_TOKEN

static final String UNKNOWN_WRITE_TOKEN

See Also:

Constant Field Values

DEFAULT_WRITE_TOKEN

static final String DEFAULT_WRITE_TOKEN

See Also:

Constant Field Values

DEFAULT_LOG_FORMAT_WRITER

static final String DEFAULT_LOG_FORMAT_WRITER

See Also:

Constant Field Values

Method Detail

newWriterBuilder

static HoodieLogFormat.WriterBuilder newWriterBuilder()

newReader

static HoodieLogFormat.Reader newReader(HoodieStorage storage,
                                        HoodieLogFile logFile,
                                        org.apache.avro.Schema readerSchema)
                                 throws IOException

Throws:

IOException

newReader

static HoodieLogFormat.Reader newReader(HoodieStorage storage,
                                        HoodieLogFile logFile,
                                        org.apache.avro.Schema readerSchema,
                                        boolean reverseReader)
                                 throws IOException

Throws:

IOException