Package org.apache.nifi.processors.hadoop.util

Class HDFSListing

java.lang.Object

org.apache.nifi.processors.hadoop.util.HDFSListing

public class HDFSListing
extends Object

A simple POJO for maintaining state about the last HDFS Listing that was performed so that we can avoid pulling the same file multiple times

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	HDFSListing.StateKeys

Field Summary

Fields

Modifier and Type	Field	Description
private Date	latestTimestamp
private Collection<String>	matchingPaths

Constructor Summary

Constructors

Constructor	Description
HDFSListing()

Method Summary

Modifier and Type	Method	Description
static HDFSListing	fromMap(Map<String,String> map)
Date	getLatestTimestamp()
Collection<String>	getMatchingPaths()
void	setLatestTimestamp(Date latestTimestamp)	Sets the timestamp of the modification date of the newest file that was contained in the HDFS Listing
void	setMatchingPaths(Collection<String> matchingPaths)	Sets the Collection containing the paths of all files in the HDFS Listing whose Modification Date was equal to getLatestTimestamp()
Map<String,String>	toMap()	Converts this HDFSListing into a Map<String, String> so that it can be stored in a StateManager.
Set<org.apache.hadoop.fs.Path>	toPaths()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

latestTimestamp

private Date latestTimestamp

matchingPaths

private Collection<String> matchingPaths

Constructor Details

HDFSListing

public HDFSListing()

Method Details

getLatestTimestamp

public Date getLatestTimestamp()

Returns:

the modification date of the newest file that was contained in the HDFS Listing

setLatestTimestamp

public void setLatestTimestamp(Date latestTimestamp)

Sets the timestamp of the modification date of the newest file that was contained in the HDFS Listing

Parameters:

latestTimestamp - the timestamp of the modification date of the newest file that was contained in the HDFS Listing

getMatchingPaths

public Collection<String> getMatchingPaths()

Returns:

a Collection containing the paths of all files in the HDFS Listing whose Modification date was equal to getLatestTimestamp()

toPaths

public Set<org.apache.hadoop.fs.Path> toPaths()

Returns:

a Collection of Path objects equivalent to those returned by getMatchingPaths()

setMatchingPaths

public void setMatchingPaths(Collection<String> matchingPaths)

Sets the Collection containing the paths of all files in the HDFS Listing whose Modification Date was equal to getLatestTimestamp()

Parameters:

matchingPaths - the paths that have last modified date matching the latest timestamp

toMap

public Map<String,String> toMap()

Converts this HDFSListing into a Map<String, String> so that it can be stored in a StateManager.

Returns:

a Map that represents the same information as this HDFSListing

fromMap

public static HDFSListing fromMap(Map<String,String> map)