org.apache.spark.streaming.dstream

Class TransformedDStream<U>

Object

org.apache.spark.streaming.dstream.DStream<U>

org.apache.spark.streaming.dstream.TransformedDStream<U>

All Implemented Interfaces:

java.io.Serializable, Logging

public class TransformedDStream<U>
extends DStream<U>

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
TransformedDStream(scala.collection.Seq<DStream<?>> parents, scala.Function2<scala.collection.Seq<RDD<?>>,Time,RDD<U>> transformFunc, scala.reflect.ClassTag<U> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
scala.Option<RDD<U>>	compute(Time validTime) Method that generates a RDD for the given time
scala.collection.immutable.List<DStream<?>>	dependencies() List of parent DStreams on which this DStream depends on
Duration	slideDuration() Time interval after which the DStream generates a RDD

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, checkpointData, checkpointDuration, clearCheckpointData, clearMetadata, context, count, countByValue, countByValueAndWindow, countByWindow, creationSite, filter, flatMap, foreach, foreach, foreachRDD, foreachRDD, generatedRDDs, generateJob, getCreationSite, getOrCompute, glom, graph, initialize, isInitialized, isTimeValid, map, mapPartitions, mustCheckpoint, parentRememberDuration, persist, persist, print, reduce, reduceByWindow, reduceByWindow, register, remember, rememberDuration, repartition, restoreCheckpointData, saveAsObjectFiles, saveAsTextFiles, setContext, setGraph, slice, slice, ssc, storageLevel, transform, transform, transformWith, transformWith, union, updateCheckpointData, validate, window, window, zeroTime

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

TransformedDStream

public TransformedDStream(scala.collection.Seq<DStream<?>> parents,
                  scala.Function2<scala.collection.Seq<RDD<?>>,Time,RDD<U>> transformFunc,
                  scala.reflect.ClassTag<U> evidence$1)

Method Detail

dependencies

public scala.collection.immutable.List<DStream<?>> dependencies()

Description copied from class: DStream

List of parent DStreams on which this DStream depends on

Specified by:

dependencies in class DStream<U>

slideDuration

public Duration slideDuration()

Description copied from class: DStream

Time interval after which the DStream generates a RDD

Specified by:

slideDuration in class DStream<U>

compute

public scala.Option<RDD<U>> compute(Time validTime)

Description copied from class: DStream

Method that generates a RDD for the given time

Specified by:

compute in class DStream<U>