org.apache.nifi.util.hive

Class HiveJdbcCommon

java.lang.Object

org.apache.nifi.util.hive.HiveJdbcCommon

public class HiveJdbcCommon
extends Object

JDBC / HiveQL common functions.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	HiveJdbcCommon.ResultSetRowCallback An interface for callback methods which allows processing of a row during the convertToXYZStream() processing.

Field Summary

Fields

Modifier and Type	Field and Description
static String	AVRO
static String	CSV
static String	CSV_MIME_TYPE
static String	MIME_TYPE_AVRO_BINARY
static PropertyDescriptor	NORMALIZE_NAMES_FOR_AVRO

Constructor Summary

Constructors

Constructor and Description
HiveJdbcCommon()

Method Summary

Modifier and Type	Method and Description
static long	convertToAvroStream(ResultSet rs, OutputStream outStream, int maxRows, boolean convertNames)
static long	convertToAvroStream(ResultSet rs, OutputStream outStream, String recordName, int maxRows, boolean convertNames, HiveJdbcCommon.ResultSetRowCallback callback)
static long	convertToCsvStream(ResultSet rs, OutputStream outStream, CsvOutputOptions outputOptions)
static long	convertToCsvStream(ResultSet rs, OutputStream outStream, String recordName, HiveJdbcCommon.ResultSetRowCallback callback, CsvOutputOptions outputOptions)
static org.apache.avro.Schema	createSchema(ResultSet rs, boolean convertNames)
static org.apache.avro.Schema	createSchema(ResultSet rs, String recordName, boolean convertNames) Creates an Avro schema from a result set.
static org.apache.hadoop.conf.Configuration	getConfigurationFromFiles(String configFiles)
static String	normalizeNameForAvro(String inputName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

AVRO

public static final String AVRO

See Also:

Constant Field Values

CSV

public static final String CSV

See Also:

Constant Field Values

MIME_TYPE_AVRO_BINARY

public static final String MIME_TYPE_AVRO_BINARY

See Also:

Constant Field Values

CSV_MIME_TYPE

public static final String CSV_MIME_TYPE

See Also:

Constant Field Values

NORMALIZE_NAMES_FOR_AVRO

public static final PropertyDescriptor NORMALIZE_NAMES_FOR_AVRO

Constructor Detail

HiveJdbcCommon

public HiveJdbcCommon()

Method Detail

convertToAvroStream

public static long convertToAvroStream(ResultSet rs,
                                       OutputStream outStream,
                                       int maxRows,
                                       boolean convertNames)
                                throws SQLException,
                                       IOException

Throws:

SQLException

IOException

convertToAvroStream

public static long convertToAvroStream(ResultSet rs,
                                       OutputStream outStream,
                                       String recordName,
                                       int maxRows,
                                       boolean convertNames,
                                       HiveJdbcCommon.ResultSetRowCallback callback)
                                throws SQLException,
                                       IOException

Throws:

SQLException

IOException

createSchema

public static org.apache.avro.Schema createSchema(ResultSet rs,
                                                  boolean convertNames)
                                           throws SQLException

Throws:

SQLException

createSchema

public static org.apache.avro.Schema createSchema(ResultSet rs,
                                                  String recordName,
                                                  boolean convertNames)
                                           throws SQLException

Creates an Avro schema from a result set. If the table/record name is known a priori and provided, use that as a fallback for the record name if it cannot be retrieved from the result set, and finally fall back to a default value.

Parameters:

rs - The result set to convert to Avro

recordName - The a priori record name to use if it cannot be determined from the result set.

convertNames - Whether to convert column/table names to be legal Avro names

Returns:

A Schema object representing the result set converted to an Avro record

Throws:

SQLException - if any error occurs during conversion

convertToCsvStream

public static long convertToCsvStream(ResultSet rs,
                                      OutputStream outStream,
                                      CsvOutputOptions outputOptions)
                               throws SQLException,
                                      IOException

Throws:

SQLException

IOException

convertToCsvStream

public static long convertToCsvStream(ResultSet rs,
                                      OutputStream outStream,
                                      String recordName,
                                      HiveJdbcCommon.ResultSetRowCallback callback,
                                      CsvOutputOptions outputOptions)
                               throws SQLException,
                                      IOException

Throws:

SQLException

IOException

normalizeNameForAvro

public static String normalizeNameForAvro(String inputName)

getConfigurationFromFiles

public static org.apache.hadoop.conf.Configuration getConfigurationFromFiles(String configFiles)