org.seqdoop.hadoop_bam

Class AnySAMInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,SAMRecordWritable>

org.seqdoop.hadoop_bam.AnySAMInputFormat

public class AnySAMInputFormat
extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,SAMRecordWritable>

An InputFormat for SAM, BAM, and CRAM files. Values are the individual records; see BAMRecordReader for the meaning of the key.

By default, files are recognized as SAM, BAM, or CRAM based on their file extensions: see TRUST_EXTS_PROPERTY. If that fails, or this behaviour is disabled, the first byte of each file is read to determine the file type.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

Field Summary

Fields

Modifier and Type	Field and Description
static String	TRUST_EXTS_PROPERTY A Boolean property: are file extensions trusted? The default is true.

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

INPUT_DIR, INPUT_DIR_RECURSIVE, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

Constructors

Constructor and Description
AnySAMInputFormat() Creates a new input format, which will use the Configuration from the first public method called.
AnySAMInputFormat(org.apache.hadoop.conf.Configuration conf) Creates a new input format, reading TRUST_EXTS_PROPERTY from the given Configuration.
AnySAMInputFormat(Map<org.apache.hadoop.fs.Path,SAMFormat> formatMap) Creates a new input format, trusting the given Map to define the file-to-format associations.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,SAMRecordWritable>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext ctx) Returns a BAMRecordReader or SAMRecordReader as appropriate, initialized with the given parameters.
SAMFormat	getFormat(org.apache.hadoop.fs.Path path) Returns the SAMFormat corresponding to the given path.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext job) Defers to BAMInputFormat or CRAMInputFormat as appropriate for each individual path.
boolean	isSplitable(org.apache.hadoop.mapreduce.JobContext job, org.apache.hadoop.fs.Path path) Defers to BAMInputFormat, CRAMInputFormat, or SAMInputFormat as appropriate for the given path.

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, listStatus, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TRUST_EXTS_PROPERTY

public static final String TRUST_EXTS_PROPERTY

A Boolean property: are file extensions trusted? The default is true.

See Also:

SAMFormat.inferFromFilePath(org.apache.hadoop.fs.Path), Constant Field Values

Constructor Detail

AnySAMInputFormat

public AnySAMInputFormat()

Creates a new input format, which will use the Configuration from the first public method called. Thus this will behave as though constructed with a Configuration directly, but only after it has received it in createRecordReader (via the TaskAttemptContext) or isSplitable or getSplits (via the JobContext). Until then, other methods will throw an IllegalStateException. This constructor exists mainly as a convenience, e.g. so that AnySAMInputFormat can be used directly in Job.setInputFormatClass.

AnySAMInputFormat

public AnySAMInputFormat(org.apache.hadoop.conf.Configuration conf)

Creates a new input format, reading TRUST_EXTS_PROPERTY from the given Configuration.

AnySAMInputFormat

public AnySAMInputFormat(Map<org.apache.hadoop.fs.Path,SAMFormat> formatMap)

Creates a new input format, trusting the given Map to define the file-to-format associations. Neither file paths nor their contents are looked at, only the Map is used.

The Map is not copied, so it should not be modified while this input format is in use!

Method Detail

getFormat

public SAMFormat getFormat(org.apache.hadoop.fs.Path path)
                    throws org.apache.hadoop.fs.PathNotFoundException

Returns the SAMFormat corresponding to the given path. Returns null if it cannot be determined even based on the file contents (unless future SAM/BAM formats are very different, this means that the path does not refer to a SAM or BAM file).

If this input format was constructed using a given Map<Path,SAMFormat> and the path is not contained within that map, throws an IllegalArgumentException.

Throws:

org.apache.hadoop.fs.PathNotFoundException

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,SAMRecordWritable> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                                        org.apache.hadoop.mapreduce.TaskAttemptContext ctx)
                                                                                                                 throws InterruptedException,
                                                                                                                        IOException

Returns a BAMRecordReader or SAMRecordReader as appropriate, initialized with the given parameters.

Throws IllegalArgumentException if the given input split is not a FileVirtualSplit (used by BAMInputFormat) or a FileSplit (used by SAMInputFormat), or if the path referred to is not recognized as a SAM, BAM, or CRAM file (see getFormat(org.apache.hadoop.fs.Path)).

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.LongWritable,SAMRecordWritable>

Throws:

InterruptedException

IOException

isSplitable

public boolean isSplitable(org.apache.hadoop.mapreduce.JobContext job,
                           org.apache.hadoop.fs.Path path)

Defers to BAMInputFormat, CRAMInputFormat, or SAMInputFormat as appropriate for the given path.

Overrides:

isSplitable in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,SAMRecordWritable>

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                       throws IOException

Defers to BAMInputFormat or CRAMInputFormat as appropriate for each individual path. SAM paths do not require special handling, so their splits are left unchanged.

Overrides:

getSplits in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.hadoop.io.LongWritable,SAMRecordWritable>

Throws:

IOException