org.apache.spark.streaming.receiver

Class ReceiverSupervisor

Object

org.apache.spark.streaming.receiver.ReceiverSupervisor

All Implemented Interfaces:

Logging

Direct Known Subclasses:

ReceiverSupervisorImpl

public abstract class ReceiverSupervisor
extends Object
implements Logging

Abstract class that is responsible for supervising a Receiver in the worker. It provides all the necessary interfaces for handling the data received by the receiver.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ReceiverSupervisor.ReceiverState

Constructor Summary

Constructors

Constructor and Description
ReceiverSupervisor(Receiver<?> receiver, SparkConf conf)

Method Summary

Methods

Modifier and Type	Method and Description
void	awaitTermination() Wait the thread until the supervisor is stopped
boolean	isReceiverStarted() Check if receiver has been marked for stopping
boolean	isReceiverStopped() Check if receiver has been marked for stopping
abstract void	pushArrayBuffer(scala.collection.mutable.ArrayBuffer<?> arrayBuffer, scala.Option<Object> optionalMetadata, scala.Option<StreamBlockId> optionalBlockId) Store an ArrayBuffer of received data as a data block into Spark's memory.
abstract void	pushBytes(java.nio.ByteBuffer bytes, scala.Option<Object> optionalMetadata, scala.Option<StreamBlockId> optionalBlockId) Store the bytes of received data as a data block into Spark's memory.
abstract void	pushIterator(scala.collection.Iterator<Object> iterator, scala.Option<Object> optionalMetadata, scala.Option<StreamBlockId> optionalBlockId) Store a iterator of received data as a data block into Spark's memory.
abstract void	pushSingle(Object data) Push a single data item to backend data store.
scala.Enumeration.Value	receiverState() State of the receiver
org.apache.spark.streaming.receiver.ReceiverSupervisor.ReceiverState$	ReceiverState()
abstract void	reportError(String message, Throwable throwable) Report errors.
void	restartReceiver(String message, scala.Option<Throwable> error) Restart receiver with delay
void	restartReceiver(String message, scala.Option<Throwable> error, int delay) Restart receiver with delay
void	start() Start the supervisor
void	startReceiver() Start receiver
void	stop(String message, scala.Option<Throwable> error) Mark the supervisor and the receiver for stopping
void	stopReceiver(String message, scala.Option<Throwable> error) Stop receiver

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

ReceiverSupervisor

public ReceiverSupervisor(Receiver<?> receiver,
                  SparkConf conf)

Method Detail

ReceiverState

public org.apache.spark.streaming.receiver.ReceiverSupervisor.ReceiverState$ ReceiverState()

receiverState

public scala.Enumeration.Value receiverState()

State of the receiver

pushSingle

public abstract void pushSingle(Object data)

Push a single data item to backend data store.

pushBytes

public abstract void pushBytes(java.nio.ByteBuffer bytes,
             scala.Option<Object> optionalMetadata,
             scala.Option<StreamBlockId> optionalBlockId)

Store the bytes of received data as a data block into Spark's memory.

pushIterator

public abstract void pushIterator(scala.collection.Iterator<Object> iterator,
                scala.Option<Object> optionalMetadata,
                scala.Option<StreamBlockId> optionalBlockId)

Store a iterator of received data as a data block into Spark's memory.

pushArrayBuffer

public abstract void pushArrayBuffer(scala.collection.mutable.ArrayBuffer<?> arrayBuffer,
                   scala.Option<Object> optionalMetadata,
                   scala.Option<StreamBlockId> optionalBlockId)

Store an ArrayBuffer of received data as a data block into Spark's memory.

reportError

public abstract void reportError(String message,
               Throwable throwable)

Report errors.

start

public void start()

Start the supervisor

stop

public void stop(String message,
        scala.Option<Throwable> error)

Mark the supervisor and the receiver for stopping

startReceiver

public void startReceiver()

Start receiver

stopReceiver

public void stopReceiver(String message,
                scala.Option<Throwable> error)

Stop receiver

restartReceiver

public void restartReceiver(String message,
                   scala.Option<Throwable> error)

Restart receiver with delay

restartReceiver

public void restartReceiver(String message,
                   scala.Option<Throwable> error,
                   int delay)

Restart receiver with delay

isReceiverStarted

public boolean isReceiverStarted()

Check if receiver has been marked for stopping

isReceiverStopped

public boolean isReceiverStopped()

Check if receiver has been marked for stopping

awaitTermination

public void awaitTermination()

Wait the thread until the supervisor is stopped