de.gerdiproject.harvest.etls.extractors

Class JsonArrayExtractor<T>

java.lang.Object

de.gerdiproject.harvest.etls.extractors.AbstractIteratorExtractor<T>

de.gerdiproject.harvest.etls.extractors.JsonArrayExtractor<T>

Type Parameters:

T - the type of elements of the JSON array

All Implemented Interfaces:

IExtractor<Iterator<T>>

public class JsonArrayExtractor<T>
extends AbstractIteratorExtractor<T>

This class represents the Extractor of an ETL process that parses a JSON array from a HTTP response.

Author:

Robin Weiss

Field Summary

Fields inherited from class de.gerdiproject.harvest.etls.extractors.AbstractIteratorExtractor

endIndex, startIndex

Constructor Summary

Constructors

Constructor and Description
JsonArrayExtractor(com.google.gson.Gson gson) Constructor that expects a JSON array to be returned directly as response from a specified URL.
JsonArrayExtractor(com.google.gson.Gson gson, String jsonQuery) Constructor that expects a JSON array to be contained in the HTTP response, navigating through the response via a specified query.

Method Summary

Modifier and Type	Method and Description
void	clear() Closes potentially open readers/writers and finishes the Extraction process if it is still ongoing.
Iterator<T>	extractAll() This method extracts everything there is to extract.
protected String	getHashFromJson(com.google.gson.JsonElement json) Retrieves a hash value from a JSON object.
protected List<T>	getListFromJson(com.google.gson.JsonElement inputJson) Parses a JSON object to retrieve a JSON array and converts the array to a LinkedList.
String	getUniqueVersionString() Returns a string that can be hashed to generate a unique identifier for the current version of the extractable elements.
String	getUrl() Retrieves the URL that is used to retrieve the JSON response.
void	init(AbstractETL<?,?> etl) Initializes the extractor for a new harvest.
void	setUrl(String url) Changes the URL that is used to retrieve the JSON response.
int	size() Returns the number of extractable documents, or -1 if they cannot be estimated.

Methods inherited from class de.gerdiproject.harvest.etls.extractors.AbstractIteratorExtractor

extract

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JsonArrayExtractor

public JsonArrayExtractor(com.google.gson.Gson gson,
                          String jsonQuery)

Constructor that expects a JSON array to be contained in the HTTP response, navigating through the response via a specified query.

Parameters:

gson - used to parse the JSON response

jsonQuery - a dot separated object structure at which the JSON array is expected

JsonArrayExtractor

public JsonArrayExtractor(com.google.gson.Gson gson)

Constructor that expects a JSON array to be returned directly as response from a specified URL.

Parameters:

gson - used to parse the JSON response

Method Detail

init

public void init(AbstractETL<?,?> etl)

Description copied from interface: IExtractor

Initializes the extractor for a new harvest.

Specified by:

init in interface IExtractor<Iterator<T>>

Overrides:

init in class AbstractIteratorExtractor<T>

Parameters:

etl - the AbstractETL to which the extractor belongs

extractAll

public Iterator<T> extractAll()

Description copied from class: AbstractIteratorExtractor

This method extracts everything there is to extract.

Specified by:

extractAll in class AbstractIteratorExtractor<T>

Returns:

an Iterator over the complete data

size

public int size()

Description copied from interface: IExtractor

Returns the number of extractable documents, or -1 if they cannot be estimated.

Specified by:

size in interface IExtractor<Iterator<T>>

Overrides:

size in class AbstractIteratorExtractor<T>

Returns:

the number of extractable documents, or -1

getUniqueVersionString

public String getUniqueVersionString()

Description copied from interface: IExtractor

Returns a string that can be hashed to generate a unique identifier for the current version of the extractable elements.

Returns:

a unique version of the extractable elements

getHashFromJson

protected String getHashFromJson(com.google.gson.JsonElement json)

Retrieves a hash value from a JSON object.

Parameters:

json - the JSON object of which the hash is retrieved

Returns:

a hash value representing the current state of the object

getListFromJson

protected List<T> getListFromJson(com.google.gson.JsonElement inputJson)
                           throws ExtractorException

Parses a JSON object to retrieve a JSON array and converts the array to a LinkedList.

Parameters:

inputJson - the JSON object of which the list is retrieved

Returns:

a list containing the elements of a JSON array

Throws:

ExtractorException - thrown when the extraction fails

clear

public void clear()

Description copied from interface: IExtractor

Closes potentially open readers/writers and finishes the Extraction process if it is still ongoing.

getUrl

public String getUrl()

Retrieves the URL that is used to retrieve the JSON response.

Returns:

the URL that is used to retrieve the JSON response

setUrl

public void setUrl(String url)

Changes the URL that is used to retrieve the JSON response.

Parameters:

url - the new URL