org.apache.tez.dag.app.dag.speculation.legacy

Class LegacyTaskRuntimeEstimator

java.lang.Object

org.apache.tez.dag.app.dag.speculation.legacy.LegacyTaskRuntimeEstimator

All Implemented Interfaces:

TaskRuntimeEstimator

public class LegacyTaskRuntimeEstimator
extends Object

Runtime estimator that uses a simple scheme of estimating task attempt runtime based on current elapsed runtime and reported progress.

Field Summary

Fields

Modifier and Type	Field and Description
protected Set<Task>	doneTasks
protected Map<org.apache.tez.dag.records.TezTaskAttemptID,Long>	startTimes
protected DataStatistics	taskStatistics
protected Vertex	vertex

Constructor Summary

Constructors

Constructor and Description
LegacyTaskRuntimeEstimator()

Method Summary

Methods

Modifier and Type	Method and Description
long	attemptEnrolledTime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)
void	contextualize(org.apache.hadoop.conf.Configuration conf, Vertex vertex)
protected DataStatistics	dataStatisticsForTask(org.apache.tez.dag.records.TezTaskID taskID)
void	enrollAttempt(org.apache.tez.dag.records.TezTaskAttemptID id, long timestamp)
long	estimatedRuntime(org.apache.tez.dag.records.TezTaskAttemptID attemptID) Estimate a task attempt's total runtime.
long	newAttemptEstimatedRuntime() Estimates how long a new attempt on this task will take if we start one now
long	runtimeEstimateVariance(org.apache.tez.dag.records.TezTaskAttemptID attemptID) Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TezTaskAttemptID)
long	thresholdRuntime(org.apache.tez.dag.records.TezTaskID taskID) Find a maximum reasonable execution wallclock time.
void	updateAttempt(org.apache.tez.dag.records.TezTaskAttemptID attemptID, TaskAttemptState state, long timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

vertex

protected Vertex vertex

startTimes

protected final Map<org.apache.tez.dag.records.TezTaskAttemptID,Long> startTimes

taskStatistics

protected final DataStatistics taskStatistics

doneTasks

protected final Set<Task> doneTasks

Constructor Detail

LegacyTaskRuntimeEstimator

public LegacyTaskRuntimeEstimator()

Method Detail

updateAttempt

public void updateAttempt(org.apache.tez.dag.records.TezTaskAttemptID attemptID,
                 TaskAttemptState state,
                 long timestamp)

Specified by:

updateAttempt in interface TaskRuntimeEstimator

estimatedRuntime

public long estimatedRuntime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

Description copied from interface: TaskRuntimeEstimator

Estimate a task attempt's total runtime. Includes the time already elapsed.

Parameters:

attemptID - the TezTaskAttemptID of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

runtimeEstimateVariance

public long runtimeEstimateVariance(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

Description copied from interface: TaskRuntimeEstimator

Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TezTaskAttemptID)

Parameters:

attemptID - the TezTaskAttemptID of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

enrollAttempt

public void enrollAttempt(org.apache.tez.dag.records.TezTaskAttemptID id,
                 long timestamp)

Specified by:

enrollAttempt in interface TaskRuntimeEstimator

attemptEnrolledTime

public long attemptEnrolledTime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

Specified by:

attemptEnrolledTime in interface TaskRuntimeEstimator

contextualize

public void contextualize(org.apache.hadoop.conf.Configuration conf,
                 Vertex vertex)

Specified by:

contextualize in interface TaskRuntimeEstimator

dataStatisticsForTask

protected DataStatistics dataStatisticsForTask(org.apache.tez.dag.records.TezTaskID taskID)

thresholdRuntime

public long thresholdRuntime(org.apache.tez.dag.records.TezTaskID taskID)

Description copied from interface: TaskRuntimeEstimator

Find a maximum reasonable execution wallclock time. Includes the time already elapsed. Find a maximum reasonable execution time. Includes the time already elapsed. If the projected total execution time for this task ever exceeds its reasonable execution time, we may speculate it.

Specified by:

thresholdRuntime in interface TaskRuntimeEstimator

Parameters:

taskID - the TezTaskID of the task we are asking about

Returns:

the task's maximum reasonable runtime, or MAX_VALUE if we don't have enough information to rule out any runtime, however long.

newAttemptEstimatedRuntime

public long newAttemptEstimatedRuntime()

Description copied from interface: TaskRuntimeEstimator

Estimates how long a new attempt on this task will take if we start one now

Specified by:

newAttemptEstimatedRuntime in interface TaskRuntimeEstimator

Returns:

our best estimate of a new attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.