org.apache.spark.broadcast

Class HttpBroadcast<T>

Object

org.apache.spark.broadcast.Broadcast<T>

org.apache.spark.broadcast.HttpBroadcast<T>

All Implemented Interfaces:

java.io.Serializable, Logging

public class HttpBroadcast<T>
extends Broadcast<T>
implements Logging, scala.Serializable

A Broadcast implementation that uses HTTP server as a broadcast mechanism. The first time a HTTP broadcast variable (sent as part of a task) is deserialized in the executor, the broadcasted data is fetched from the driver (through a HTTP server running at the driver) and stored in the BlockManager of the executor to speed up future accesses.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HttpBroadcast(T value_, boolean isLocal, long id, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
static java.io.File	getFile(long id)
static void	initialize(boolean isDriver, SparkConf conf, SecurityManager securityMgr)
static void	stop()
static void	unpersist(long id, boolean removeFromDriver, boolean blocking) Remove all persisted blocks associated with this HTTP broadcast on the executors.
T	value_()

Methods inherited from class org.apache.spark.broadcast.Broadcast

destroy, destroy, id, isValid, toString, unpersist, unpersist, value

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

HttpBroadcast

public HttpBroadcast(T value_,
             boolean isLocal,
             long id,
             scala.reflect.ClassTag<T> evidence$1)

Method Detail

initialize

public static void initialize(boolean isDriver,
              SparkConf conf,
              SecurityManager securityMgr)

stop

public static void stop()

getFile

public static java.io.File getFile(long id)

unpersist

public static void unpersist(long id,
             boolean removeFromDriver,
             boolean blocking)

Remove all persisted blocks associated with this HTTP broadcast on the executors. If removeFromDriver is true, also remove these persisted blocks on the driver and delete the associated broadcast file.

value_

public T value_()