org.apache.parquet.cli

Class BaseCommand

java.lang.Object

org.apache.parquet.cli.BaseCommand

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, Command

Direct Known Subclasses:

CatCommand, CheckParquet251Command, ColumnMaskingCommand, ColumnSizeCommand, ConvertCommand, ConvertCSVCommand, CSVSchemaCommand, ParquetMetadataCommand, PruneColumnsCommand, RewriteCommand, ScanCommand, SchemaCommand, ShowBloomFilterCommand, ShowColumnIndexCommand, ShowDictionaryCommand, ShowFooterCommand, ShowPagesCommand, ToAvroCommand, TransCompressionCommand

public abstract class BaseCommand
extends Object
implements Command, org.apache.hadoop.conf.Configurable

Field Summary

Fields

Modifier and Type	Field and Description
protected org.slf4j.Logger	console

Constructor Summary

Constructors

Constructor and Description
BaseCommand(org.slf4j.Logger console)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.FSDataOutputStream	create(String filename) Creates a file and returns an open FSDataOutputStream.
org.apache.hadoop.fs.FSDataOutputStream	createWithChecksum(String filename) Creates a file and returns an open FSDataOutputStream.
org.apache.hadoop.fs.FSDataOutputStream	createWithNoOverwrite(String filename) Creates a file and returns an open FSDataOutputStream.
org.apache.hadoop.fs.FileSystem	defaultFS()
protected org.apache.avro.Schema	getAvroSchema(String source)
org.apache.hadoop.conf.Configuration	getConf()
protected static ClassLoader	loaderFor(List<String> jars, List<String> paths) Returns a ClassLoader for a set of jars and directories.
protected static ClassLoader	loaderForJars(List<String> jars) Returns a ClassLoader for a set of jars.
protected static ClassLoader	loaderForPaths(List<String> paths) Returns a ClassLoader for a set of directories.
InputStream	open(String filename) Opens an existing file or resource.
protected <D> Iterable<D>	openDataFile(String source, org.apache.avro.Schema projection)
org.apache.avro.file.SeekableInput	openSeekable(String filename)
void	output(String content, org.slf4j.Logger console, String filename) Output content to the console or a file.
org.apache.hadoop.fs.Path	qualifiedPath(String filename) Returns a qualified Path for the filename.
URI	qualifiedURI(String filename) Returns a URI for the filename that is a qualified Path or a resource URI.
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.parquet.cli.Command

getExamples, run

Field Detail

console

protected final org.slf4j.Logger console

Constructor Detail

BaseCommand

public BaseCommand(org.slf4j.Logger console)

Method Detail

defaultFS

public org.apache.hadoop.fs.FileSystem defaultFS()
                                          throws IOException

Returns:

FileSystem to use when no file system scheme is present in a path

Throws:

IOException - if there is an error loading the default fs

output

public void output(String content,
                   org.slf4j.Logger console,
                   String filename)
            throws IOException

Output content to the console or a file. This will not produce checksum files.

Parameters:

content - String content to write

console - A Logger for writing to the console

filename - The destination Path as a String

Throws:

IOException - if there is an error while writing

create

public org.apache.hadoop.fs.FSDataOutputStream create(String filename)
                                               throws IOException

Creates a file and returns an open FSDataOutputStream. If the file does not have a file system scheme, this uses the default FS. This will not produce checksum files and will overwrite a file that already exists.

Parameters:

filename - The filename to create

Returns:

An open FSDataOutputStream

Throws:

IOException - if there is an error creating the file

createWithChecksum

public org.apache.hadoop.fs.FSDataOutputStream createWithChecksum(String filename)
                                                           throws IOException

Creates a file and returns an open FSDataOutputStream. If the file does not have a file system scheme, this uses the default FS. This will produce checksum files and will overwrite a file that already exists.

Parameters:

filename - The filename to create

Returns:

An open FSDataOutputStream

Throws:

IOException - if there is an error creating the file

createWithNoOverwrite

public org.apache.hadoop.fs.FSDataOutputStream createWithNoOverwrite(String filename)
                                                              throws IOException

Creates a file and returns an open FSDataOutputStream. If the file does not have a file system scheme, this uses the default FS. This will neither produce checksum files nor overwrite a file that already exists.

Parameters:

filename - The filename to create

Returns:

An open FSDataOutputStream

Throws:

IOException - if there is an error creating the file

qualifiedPath

public org.apache.hadoop.fs.Path qualifiedPath(String filename)
                                        throws IOException

Returns a qualified Path for the filename. If the file does not have a file system scheme, this uses the default FS.

Parameters:

filename - The filename to qualify

Returns:

A qualified Path for the filename

Throws:

IOException - if there is an error creating a qualified path

qualifiedURI

public URI qualifiedURI(String filename)
                 throws IOException

Returns a URI for the filename that is a qualified Path or a resource URI. If the file does not have a file system scheme, this uses the default FS.

Parameters:

filename - The filename to qualify

Returns:

A qualified URI for the filename

Throws:

IOException - if there is an error creating a qualified URI

open

public InputStream open(String filename)
                 throws IOException

Opens an existing file or resource. If the file does not have a file system scheme, this uses the default FS.

Parameters:

filename - The filename to open.

Returns:

An open InputStream with the file contents

Throws:

IOException - if there is an error opening the file

IllegalArgumentException - If the file does not exist

openSeekable

public org.apache.avro.file.SeekableInput openSeekable(String filename)
                                                throws IOException

Throws:

IOException

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

loaderFor

protected static ClassLoader loaderFor(List<String> jars,
                                       List<String> paths)
                                throws MalformedURLException

Returns a ClassLoader for a set of jars and directories.

Parameters:

jars - A list of jar paths

paths - A list of directories containing .class files

Returns:

a classloader for the jars and paths

Throws:

MalformedURLException - if a jar or path is invalid

loaderForJars

protected static ClassLoader loaderForJars(List<String> jars)
                                    throws MalformedURLException

Returns a ClassLoader for a set of jars.

Parameters:

jars - A list of jar paths

Returns:

a classloader for the jars

Throws:

MalformedURLException - if a URL is invalid

loaderForPaths

protected static ClassLoader loaderForPaths(List<String> paths)
                                     throws MalformedURLException

Returns a ClassLoader for a set of directories.

Parameters:

paths - A list of directories containing .class files

Returns:

a classloader for the paths

Throws:

MalformedURLException - if a path is invalid

openDataFile

protected <D> Iterable<D> openDataFile(String source,
                                       org.apache.avro.Schema projection)
                                throws IOException

Throws:

IOException

getAvroSchema

protected org.apache.avro.Schema getAvroSchema(String source)
                                        throws IOException

Throws:

IOException