org.apache.spark.streaming.scheduler

BatchInfo

case class BatchInfo(batchTime: Time, streamIdToNumRecords: Map[Int, Long], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long]) extends Product with Serializable

:: DeveloperApi :: Class having information on completed batches.

batchTime

Time of the batch

streamIdToNumRecords

A map of input stream id to record number

submissionTime

Clock time of when jobs of this batch was submitted to the streaming scheduler queue

processingStartTime

Clock time of when the first job of this batch started processing

processingEndTime

Clock time of when the last job of this batch finished processing

Annotations
@DeveloperApi()
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. BatchInfo
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BatchInfo(batchTime: Time, streamIdToNumRecords: Map[Int, Long], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long])

    batchTime

    Time of the batch

    streamIdToNumRecords

    A map of input stream id to record number

    submissionTime

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

    processingStartTime

    Clock time of when the first job of this batch started processing

    processingEndTime

    Clock time of when the last job of this batch finished processing

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  7. val batchTime: Time

    Time of the batch

  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  10. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  11. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  12. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  13. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  14. final def notify(): Unit

    Definition Classes
    AnyRef

  15. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  16. def numRecords: Long

    The number of recorders received by the receivers in this batch.

  17. def processingDelay: Option[Long]

    Time taken for the all jobs of this batch to finish processing from the time they started processing.

    Time taken for the all jobs of this batch to finish processing from the time they started processing. Essentially, it is processingEndTime - processingStartTime.

  18. val processingEndTime: Option[Long]

    Clock time of when the last job of this batch finished processing

  19. val processingStartTime: Option[Long]

    Clock time of when the first job of this batch started processing

  20. def schedulingDelay: Option[Long]

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler.

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler. Essentially, it is processingStartTime - submissionTime.

  21. val streamIdToNumRecords: Map[Int, Long]

    A map of input stream id to record number

  22. val submissionTime: Long

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

  23. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  24. def totalDelay: Option[Long]

    Time taken for all the jobs of this batch to finish processing from the time they were submitted.

    Time taken for all the jobs of this batch to finish processing from the time they were submitted. Essentially, it is processingDelay + schedulingDelay.

  25. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  27. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped