org.apache.spark

Class TaskContext

Object

org.apache.spark.TaskContext

All Implemented Interfaces:

java.io.Serializable

public abstract class TaskContext
extends Object
implements java.io.Serializable

Contextual information about a task which can be read or mutated during execution. To access the TaskContext for a running task, use:

   org.apache.spark.TaskContext.get()
 

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
TaskContext()

Method Summary

Methods

Modifier and Type	Method and Description
abstract void	addOnCompleteCallback(scala.Function0<scala.runtime.BoxedUnit> f) Adds a callback function to be executed on task completion.
TaskContext	addTaskCompletionListener(scala.Function1<TaskContext,scala.runtime.BoxedUnit> f) Adds a listener in the form of a Scala closure to be executed on task completion.
abstract TaskContext	addTaskCompletionListener(TaskCompletionListener listener) Adds a (Java friendly) listener to be executed on task completion.
TaskContext	addTaskFailureListener(scala.Function2<TaskContext,Throwable,scala.runtime.BoxedUnit> f) Adds a listener to be executed on task failure.
abstract TaskContext	addTaskFailureListener(TaskFailureListener listener) Adds a listener to be executed on task failure.
abstract long	attemptId()
abstract int	attemptNumber() How many times this task has been attempted.
static TaskContext	get() Return the currently active TaskContext.
abstract scala.collection.Seq<org.apache.spark.metrics.source.Source>	getMetricsSources(String sourceName) ::DeveloperApi:: Returns all metrics sources with the given name which are associated with the instance which runs the task.
static int	getPartitionId() Returns the partition id of currently active TaskContext.
abstract scala.collection.immutable.Map<String,Accumulator<Object>>	internalMetricsToAccumulators() Accumulators for tracking internal metrics indexed by the name.
abstract boolean	isCompleted() Returns true if the task has completed.
abstract boolean	isInterrupted() Returns true if the task has been killed.
abstract boolean	isRunningLocally() Returns true if the task is running locally in the driver program.
abstract int	partitionId() The ID of the RDD partition that is computed by this task.
abstract boolean	runningLocally()
abstract int	stageId() The ID of the stage that this task belong to.
abstract long	taskAttemptId() An ID that is unique to this task attempt (within the same SparkContext, no two task attempts will share the same attempt ID).
abstract org.apache.spark.executor.TaskMetrics	taskMetrics() ::DeveloperApi::

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TaskContext

public TaskContext()

Method Detail

get

public static TaskContext get()

Return the currently active TaskContext. This can be called inside of user functions to access contextual information about running tasks.

Returns:

(undocumented)

getPartitionId

public static int getPartitionId()

Returns the partition id of currently active TaskContext. It will return 0 if there is no active TaskContext for cases like local execution.

Returns:

(undocumented)

isCompleted

public abstract boolean isCompleted()

Returns true if the task has completed.

Returns:

(undocumented)

isInterrupted

public abstract boolean isInterrupted()

Returns true if the task has been killed.

Returns:

(undocumented)

runningLocally

public abstract boolean runningLocally()

isRunningLocally

public abstract boolean isRunningLocally()

Returns true if the task is running locally in the driver program.

Returns:

addTaskCompletionListener

public abstract TaskContext addTaskCompletionListener(TaskCompletionListener listener)

Adds a (Java friendly) listener to be executed on task completion. This will be called in all situation - success, failure, or cancellation. An example use is for HadoopRDD to register a callback to close the input stream.

Exceptions thrown by the listener will result in failure of the task.

Parameters:

listener - (undocumented)

Returns:

(undocumented)

addTaskCompletionListener

public TaskContext addTaskCompletionListener(scala.Function1<TaskContext,scala.runtime.BoxedUnit> f)

Adds a listener in the form of a Scala closure to be executed on task completion. This will be called in all situations - success, failure, or cancellation. An example use is for HadoopRDD to register a callback to close the input stream.

Exceptions thrown by the listener will result in failure of the task.

Parameters:

f - (undocumented)

Returns:

(undocumented)

addTaskFailureListener

public abstract TaskContext addTaskFailureListener(TaskFailureListener listener)

Adds a listener to be executed on task failure. Operations defined here must be idempotent, as onTaskFailure can be called multiple times.

Parameters:

listener - (undocumented)

Returns:

(undocumented)

addTaskFailureListener

public TaskContext addTaskFailureListener(scala.Function2<TaskContext,Throwable,scala.runtime.BoxedUnit> f)

Adds a listener to be executed on task failure. Operations defined here must be idempotent, as onTaskFailure can be called multiple times.

Parameters:

f - (undocumented)

Returns:

(undocumented)

addOnCompleteCallback

public abstract void addOnCompleteCallback(scala.Function0<scala.runtime.BoxedUnit> f)

Adds a callback function to be executed on task completion. An example use is for HadoopRDD to register a callback to close the input stream. Will be called in any situation - success, failure, or cancellation.

Parameters:

f - Callback function.

stageId

public abstract int stageId()

The ID of the stage that this task belong to.

Returns:

(undocumented)

partitionId

public abstract int partitionId()

The ID of the RDD partition that is computed by this task.

Returns:

(undocumented)

attemptNumber

public abstract int attemptNumber()

How many times this task has been attempted. The first task attempt will be assigned attemptNumber = 0, and subsequent attempts will have increasing attempt numbers.

Returns:

(undocumented)

attemptId

public abstract long attemptId()

taskAttemptId

public abstract long taskAttemptId()

An ID that is unique to this task attempt (within the same SparkContext, no two task attempts will share the same attempt ID). This is roughly equivalent to Hadoop's TaskAttemptID.

Returns:

(undocumented)

taskMetrics

public abstract org.apache.spark.executor.TaskMetrics taskMetrics()

::DeveloperApi::

getMetricsSources

public abstract scala.collection.Seq<org.apache.spark.metrics.source.Source> getMetricsSources(String sourceName)

::DeveloperApi:: Returns all metrics sources with the given name which are associated with the instance which runs the task. For more information see org.apache.spark.metrics.MetricsSystem!.

Parameters:

sourceName - (undocumented)

Returns:

(undocumented)

internalMetricsToAccumulators

public abstract scala.collection.immutable.Map<String,Accumulator<Object>> internalMetricsToAccumulators()

Accumulators for tracking internal metrics indexed by the name.

Returns:

(undocumented)