org.apache.spark.mllib.rdd

Class RDDFunctions<T>

java.lang.Object

org.apache.spark.mllib.rdd.RDDFunctions<T>

All Implemented Interfaces:

java.io.Serializable

public class RDDFunctions<T>
extends java.lang.Object
implements scala.Serializable

Machine learning specific RDD functions.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RDDFunctions(RDD<T> self, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
static <T> RDDFunctions<T>	fromRDD(RDD<T> rdd, scala.reflect.ClassTag<T> evidence$3) Implicit conversion from an RDD to RDDFunctions.
RDD<java.lang.Object>	sliding(int windowSize) sliding(Int, Int)* with step = 1.
RDD<java.lang.Object>	sliding(int windowSize, int step) Returns a RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them.
<U> U	treeAggregate(U zeroValue, scala.Function2<U,T,U> seqOp, scala.Function2<U,U,U> combOp, int depth, scala.reflect.ClassTag<U> evidence$2) Deprecated. Use RDD.treeAggregate(U, scala.Function2<U, T, U>, scala.Function2<U, U, U>, int, scala.reflect.ClassTag<U>) instead.
T	treeReduce(scala.Function2<T,T,T> f, int depth) Deprecated. Use RDD.treeReduce(scala.Function2<T, T, T>, int) instead.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RDDFunctions

public RDDFunctions(RDD<T> self,
            scala.reflect.ClassTag<T> evidence$1)

Method Detail

fromRDD

public static <T> RDDFunctions<T> fromRDD(RDD<T> rdd,
                          scala.reflect.ClassTag<T> evidence$3)

Implicit conversion from an RDD to RDDFunctions.

sliding

public RDD<java.lang.Object> sliding(int windowSize,
                            int step)

Returns a RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them. The ordering is first based on the partition index and then the ordering of items within each partition. This is similar to sliding in Scala collections, except that it becomes an empty RDD if the window size is greater than the total number of items. It needs to trigger a Spark job if the parent RDD has more than one partitions and the window size is greater than 1.

Parameters:

windowSize - (undocumented)

step - (undocumented)

Returns:

(undocumented)

sliding

public RDD<java.lang.Object> sliding(int windowSize)

sliding(Int, Int)* with step = 1.

Parameters:

windowSize - (undocumented)

Returns:

(undocumented)

treeReduce

public T treeReduce(scala.Function2<T,T,T> f,
           int depth)

Deprecated. Use RDD.treeReduce(scala.Function2<T, T, T>, int) instead.

Reduces the elements of this RDD in a multi-level tree pattern.

Parameters:

depth - suggested depth of the tree (default: 2)

f - (undocumented)

Returns:

(undocumented)

See Also:

RDD.treeReduce(scala.Function2<T, T, T>, int)

treeAggregate

public <U> U treeAggregate(U zeroValue,
                  scala.Function2<U,T,U> seqOp,
                  scala.Function2<U,U,U> combOp,
                  int depth,
                  scala.reflect.ClassTag<U> evidence$2)

Deprecated. Use RDD.treeAggregate(U, scala.Function2<U, T, U>, scala.Function2<U, U, U>, int, scala.reflect.ClassTag<U>) instead.

Aggregates the elements of this RDD in a multi-level tree pattern.

Parameters:

depth - suggested depth of the tree (default: 2)

zeroValue - (undocumented)

seqOp - (undocumented)

combOp - (undocumented)

evidence$2 - (undocumented)

Returns:

(undocumented)

See Also:

RDD.treeAggregate(U, scala.Function2<U, T, U>, scala.Function2<U, U, U>, int, scala.reflect.ClassTag<U>)