org.apache.spark.streaming.dstream

Class MapWithStateDStream<KeyType,ValueType,StateType,MappedType>

Object

org.apache.spark.streaming.dstream.DStream<MappedType>

org.apache.spark.streaming.dstream.MapWithStateDStream<KeyType,ValueType,StateType,MappedType>

Type Parameters:

KeyType - Class of the key

ValueType - Class of the value

StateType - Class of the state data

MappedType - Class of the mapped data

All Implemented Interfaces:

java.io.Serializable, Logging

public abstract class MapWithStateDStream<KeyType,ValueType,StateType,MappedType>
extends DStream<MappedType>

:: Experimental :: DStream representing the stream of data generated by mapWithState operation on a pair DStream. Additionally, it also gives access to the stream of state snapshots, that is, the state data of all keys after a batch has updated them.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
MapWithStateDStream(StreamingContext ssc, scala.reflect.ClassTag<MappedType> evidence$1)

Method Summary

Modifier and Type	Method and Description
abstract DStream<scala.Tuple2<KeyType,StateType>>	stateSnapshots() Return a pair DStream where each RDD is the snapshot of the state of all the keys.

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, compute, context, count, countByValue, countByValueAndWindow, countByWindow, dependencies, filter, flatMap, foreachRDD, foreachRDD, glom, map, mapPartitions, persist, persist, print, print, reduce, reduceByWindow, reduceByWindow, repartition, saveAsObjectFiles, saveAsTextFiles, slice, slice, slideDuration, toPairDStreamFunctions, transform, transform, transformWith, transformWith, union, window, window

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

MapWithStateDStream

public MapWithStateDStream(StreamingContext ssc,
                           scala.reflect.ClassTag<MappedType> evidence$1)

Method Detail

stateSnapshots

public abstract DStream<scala.Tuple2<KeyType,StateType>> stateSnapshots()

Return a pair DStream where each RDD is the snapshot of the state of all the keys.