de.gerdiproject.harvest.etls.loaders

Class ElasticSearchLoader

java.lang.Object

de.gerdiproject.harvest.etls.loaders.AbstractIteratorLoader<S>

de.gerdiproject.harvest.etls.loaders.AbstractURLLoader<de.gerdiproject.json.datacite.DataCiteJson>

de.gerdiproject.harvest.etls.loaders.ElasticSearchLoader

All Implemented Interfaces:

ILoader<Iterator<de.gerdiproject.json.datacite.DataCiteJson>>

public class ElasticSearchLoader
extends AbstractURLLoader<de.gerdiproject.json.datacite.DataCiteJson>

This class serves as a communicator for an Elastic Search node. An URL and optionally a username and password must be set up prior to the loader execution.

Author:

Robin Weiss

Field Summary

Fields inherited from class de.gerdiproject.harvest.etls.loaders.AbstractURLLoader

batchMap, charset, hashGenerator, logger, maxBatchSizeParam, urlParam

Fields inherited from class de.gerdiproject.harvest.etls.loaders.AbstractIteratorLoader

dedicatedEtl, hasLoadedDocuments

Constructor Summary

Constructors

Constructor and Description
ElasticSearchLoader() Constructor that initializes a Json parser for server responses.

Method Summary

Modifier and Type	Method and Description
protected String	getCredentials() Retrieves the credentials that may be necessary for to authenticate the loader with the URL.
protected int	getSizeOfDocument(String documentId, de.gerdiproject.harvest.IDocument document) Calculates the size of a single document within the batch in bytes.
protected String	getUrl() Returns the loader target URL as a string.
protected void	loadBatch(Map<String,de.gerdiproject.harvest.IDocument> documents) The core of the loading function which is to be implemented by subclasses.

Methods inherited from class de.gerdiproject.harvest.etls.loaders.AbstractURLLoader

checkPreconditionErrors, clear, init, load, loadElement, tryLoadingBatch, unregisterParameters

Methods inherited from class de.gerdiproject.harvest.etls.loaders.AbstractIteratorLoader

loadElementAndIncrement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ElasticSearchLoader

public ElasticSearchLoader()

Constructor that initializes a Json parser for server responses.

Method Detail

loadBatch

protected void loadBatch(Map<String,de.gerdiproject.harvest.IDocument> documents)
                  throws Exception

Description copied from class: AbstractURLLoader

The core of the loading function which is to be implemented by subclasses. This method should load each document of the map to the destination URL via a REST request.

Specified by:

loadBatch in class AbstractURLLoader<de.gerdiproject.json.datacite.DataCiteJson>

Parameters:

documents - a map of documentIDs to documents that are to be loaded, the values may also be null, in which case the document is to be removed from the index

Throws:

Exception - any kind of exception that can be thrown by the loading process

getUrl

protected String getUrl()

Description copied from class: AbstractURLLoader

Returns the loader target URL as a string.

Overrides:

getUrl in class AbstractURLLoader<de.gerdiproject.json.datacite.DataCiteJson>

Returns:

the loader target URL as a string

getCredentials

protected String getCredentials()

Description copied from class: AbstractURLLoader

Retrieves the credentials that may be necessary for to authenticate the loader with the URL.

Overrides:

getCredentials in class AbstractURLLoader<de.gerdiproject.json.datacite.DataCiteJson>

Returns:

the credentials that may be necessary to authenticate the loader with the URL

getSizeOfDocument

protected int getSizeOfDocument(String documentId,
                                de.gerdiproject.harvest.IDocument document)

Description copied from class: AbstractURLLoader

Calculates the size of a single document within the batch in bytes.

Specified by:

getSizeOfDocument in class AbstractURLLoader<de.gerdiproject.json.datacite.DataCiteJson>

Parameters:

documentId - the unique identifier of the document

document - the document of which the size is measured

Returns:

the size of the document in bytes