org.apache.solr.hadoop
Class SolrCloudPartitioner

java.lang.Object
  org.apache.hadoop.mapreduce.Partitioner<org.apache.hadoop.io.Text,SolrInputDocumentWritable>
      org.apache.solr.hadoop.SolrCloudPartitioner

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class SolrCloudPartitioner
extends org.apache.hadoop.mapreduce.Partitioner<org.apache.hadoop.io.Text,SolrInputDocumentWritable>
implements org.apache.hadoop.conf.Configurable

MapReduce partitioner that partitions the Mapper output such that each SolrInputDocument gets sent to the SolrCloud shard that it would have been sent to if the document were ingested via the standard SolrCloud Near Real Time (NRT) API. In other words, this class implements the same partitioning semantics as the standard SolrCloud NRT API. This enables to mix batch updates from MapReduce ingestion with updates from standard NRT ingestion on the same SolrCloud cluster, using identical unique document keys.

Field Summary

static String	COLLECTION
static String	SHARDS
static String	ZKHOST

Constructor Summary

SolrCloudPartitioner()

Method Summary

org.apache.hadoop.conf.Configuration	getConf()
int	getPartition(org.apache.hadoop.io.Text key, SolrInputDocumentWritable value, int numPartitions)
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SHARDS

public static final String SHARDS

ZKHOST

public static final String ZKHOST

COLLECTION

public static final String COLLECTION

Constructor Detail

SolrCloudPartitioner

public SolrCloudPartitioner()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

getPartition

public int getPartition(org.apache.hadoop.io.Text key,
                        SolrInputDocumentWritable value,
                        int numPartitions)

Specified by:

getPartition in class org.apache.hadoop.mapreduce.Partitioner<org.apache.hadoop.io.Text,SolrInputDocumentWritable>