org.apache.beam.sdk.testing

Class TestPipeline

java.lang.Object

org.apache.beam.sdk.Pipeline

org.apache.beam.sdk.testing.TestPipeline

public class TestPipeline
extends Pipeline

A creator of test pipelines that can be used inside of tests that can be configured to run locally or against a remote pipeline runner.

It is recommended to tag hand-selected tests for this purpose using the RunnableOnService Category annotation, as each test run against a pipeline runner will utilize resources of that pipeline runner.

In order to run tests on a pipeline runner, the following conditions must be met:

• System property "beamTestPipelineOptions" must contain a JSON delimited list of pipeline options. For example:

   [
       "--runner=org.apache.beam.runners.dataflow.testing.TestDataflowRunner",
       "--project=mygcpproject",
       "--stagingLocation=gs://mygcsbucket/path"
       ]

  Note that the set of pipeline options required is pipeline runner specific.
• Jars containing the SDK and test classes must be available on the classpath.

Use PAssert for tests, as it integrates with this test harness in both direct and remote execution modes. For example:

 Pipeline p = TestPipeline.create();
 PCollection<Integer> output = ...

 PAssert.that(output)
     .containsInAnyOrder(1, 2, 3, 4);
 p.run();
 

For pipeline runners, it is required that they must throw an AssertionError containing the message from the PAssert that failed.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.Pipeline

Pipeline.PipelineExecutionException, Pipeline.PipelineVisitor

Method Summary

Modifier and Type	Method and Description
static String[]	convertToArgs(PipelineOptions options)
static TestPipeline	create() Creates and returns a new test pipeline.
static TestPipeline	fromOptions(PipelineOptions options)
PipelineResult	run() Runs this TestPipeline, unwrapping any AssertionError that is raised during testing.
static PipelineOptions	testingPipelineOptions() Creates PipelineOptions for testing.
String	toString()

Methods inherited from class org.apache.beam.sdk.Pipeline

addValueInternal, apply, apply, applyTransform, applyTransform, begin, create, getCoderRegistry, getFullNameForTesting, getOptions, getRunner, setCoderRegistry, traverseTopologically

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

create

public static TestPipeline create()

Creates and returns a new test pipeline.

Use PAssert to add tests, then call Pipeline.run() to execute the pipeline and check the tests.

fromOptions

public static TestPipeline fromOptions(PipelineOptions options)

run

public PipelineResult run()

Runs this TestPipeline, unwrapping any AssertionError that is raised during testing.

Overrides:

run in class Pipeline

toString

public String toString()

Overrides:

toString in class Pipeline

testingPipelineOptions

public static PipelineOptions testingPipelineOptions()

Creates PipelineOptions for testing.

convertToArgs

public static String[] convertToArgs(PipelineOptions options)