org.apache.beam.sdk.io.synthetic

Class SyntheticOptions

java.lang.Object

org.apache.beam.sdk.io.synthetic.SyntheticOptions

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

SyntheticSourceOptions, SyntheticStep.Options

public class SyntheticOptions
extends java.lang.Object
implements java.io.Serializable

This SyntheticOptions class provides common parameterizable synthetic options that are used by SyntheticBoundedSource and SyntheticUnboundedSource.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SyntheticOptions.DelayType The type of Delay that will be produced.
static interface	SyntheticOptions.Sampler Wrapper over a distribution.

Field Summary

Fields

Modifier and Type	Field and Description
long	bytesPerRecord The size of a single record used for size estimation in bytes.
double	cpuUtilizationInMixedDelay CPU utilization when delayType is 'MIXED'.
SyntheticOptions.DelayType	delayType When 'delayDistribution' is configured, this indicates how the delay enforced ("SLEEP", "CPU", or "MIXED").
double	hotKeyFraction The fraction of records associated with "hot" keys, which are uniformly distributed over a fixed number of hot keys.
long	keySizeBytes The key size in bytes.
double	largeKeyFraction The fraction of keys that should be larger than others.
double	largeKeySizeBytes The size of large keys.
long	numHotKeys The number of distinct "hot" keys.
int	seed The seed is used for generating a hash function implementing the 128-bit murmur3 algorithm.
long	valueSizeBytes The value size in bytes.

Method Summary

Modifier and Type	Method and Description
static SyntheticOptions.Sampler	fromIntegerDistribution(org.apache.commons.math3.distribution.IntegerDistribution dist)
static <T extends SyntheticOptions> T	fromJsonString(java.lang.String json, java.lang.Class<T> type)
static SyntheticOptions.Sampler	fromRealDistribution(org.apache.commons.math3.distribution.RealDistribution dist)
org.apache.beam.sdk.values.KV<byte[],byte[]>	genKvPair(long seed)
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.hash.HashFunction	hashFunction()
long	nextDelay(long seed)
void	setSeed(int seed)
java.lang.String	toString()
void	validate()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

keySizeBytes

public long keySizeBytes

The key size in bytes.

valueSizeBytes

public long valueSizeBytes

The value size in bytes.

bytesPerRecord

public final long bytesPerRecord

The size of a single record used for size estimation in bytes. If less than zero, keySizeBytes + valueSizeBytes is used.

numHotKeys

public long numHotKeys

The number of distinct "hot" keys.

hotKeyFraction

public double hotKeyFraction

The fraction of records associated with "hot" keys, which are uniformly distributed over a fixed number of hot keys.

largeKeyFraction

public double largeKeyFraction

The fraction of keys that should be larger than others.

largeKeySizeBytes

public double largeKeySizeBytes

The size of large keys.

seed

public int seed

The seed is used for generating a hash function implementing the 128-bit murmur3 algorithm.

delayType

public final SyntheticOptions.DelayType delayType

When 'delayDistribution' is configured, this indicates how the delay enforced ("SLEEP", "CPU", or "MIXED").

cpuUtilizationInMixedDelay

public final double cpuUtilizationInMixedDelay

CPU utilization when delayType is 'MIXED'. This determines the fraction of processing time spent spinning. The remaining time is spent sleeping. For each millisecond of processing time we choose to spin with probability equal to this fraction.

Method Detail

fromRealDistribution

public static SyntheticOptions.Sampler fromRealDistribution(org.apache.commons.math3.distribution.RealDistribution dist)

fromIntegerDistribution

public static SyntheticOptions.Sampler fromIntegerDistribution(org.apache.commons.math3.distribution.IntegerDistribution dist)

setSeed

public void setSeed(int seed)

hashFunction

public org.apache.beam.vendor.guava.v26_0_jre.com.google.common.hash.HashFunction hashFunction()

validate

public void validate()

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

nextDelay

public long nextDelay(long seed)

genKvPair

public org.apache.beam.sdk.values.KV<byte[],byte[]> genKvPair(long seed)

fromJsonString

public static <T extends SyntheticOptions> T fromJsonString(java.lang.String json,
                                                            java.lang.Class<T> type)
                                                     throws java.io.IOException

Throws:

java.io.IOException