LocalLDAModel

java.lang.Object
- org.apache.spark.mllib.clustering.LDAModel
- - org.apache.spark.mllib.clustering.LocalLDAModel

All Implemented Interfaces:

java.io.Serializable, Saveable
```
public class LocalLDAModel
extends LDAModel
implements scala.Serializable
```
Local LDA model. This model stores only the inferred topics.
param: topics Inferred topics (vocabSize x k matrix).

See Also:
Serialized Form

Method Summary

Methods
Modifier and Type	Method and Description
`scala.Tuple2<int[],double[]>[]`	`describeTopics(int maxTermsPerTopic)` Return the topics described by weighted terms.
`Vector`	`docConcentration()` Concentration parameter (commonly named "alpha") for the prior placed on documents' distributions over topics ("theta").
`protected java.lang.String`	`formatVersion()` Current version of model save/load format.
`protected double`	`gammaShape()` Shape parameter for random initialization of variational parameter gamma.
`int`	`k()` Number of topics
`static LocalLDAModel`	`load(SparkContext sc, java.lang.String path)`
`double`	`logLikelihood(JavaPairRDD<java.lang.Long,Vector> documents)` Java-friendly version of `logLikelihood`
`double`	`logLikelihood(RDD<scala.Tuple2<java.lang.Object,Vector>> documents)` Calculates a lower bound on the log likelihood of the entire corpus.
`double`	`logPerplexity(JavaPairRDD<java.lang.Long,Vector> documents)` Java-friendly version of `logPerplexity`
`double`	`logPerplexity(RDD<scala.Tuple2<java.lang.Object,Vector>> documents)` Calculate an upper bound bound on perplexity.
`void`	`save(SparkContext sc, java.lang.String path)` Save this model to the given path.
`double`	`topicConcentration()` Concentration parameter (commonly named "beta" or "eta") for the prior placed on topics' distributions over terms.
`JavaPairRDD<java.lang.Long,Vector>`	`topicDistributions(JavaPairRDD<java.lang.Long,Vector> documents)` Java-friendly version of `topicDistributions`
`RDD<scala.Tuple2<java.lang.Object,Vector>>`	`topicDistributions(RDD<scala.Tuple2<java.lang.Object,Vector>> documents)` Predicts the topic mixture distribution for each document (often called "theta" in the literature).
`Matrix`	`topics()`
`Matrix`	`topicsMatrix()` Inferred topics, where each topic is represented by a distribution over terms.
`int`	`vocabSize()` Vocabulary size (number of terms or terms in the vocabulary)

Methods inherited from class org.apache.spark.mllib.clustering.LDAModel
describeTopics

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Method Detail
  - load
```
public static LocalLDAModel load(SparkContext sc,
                 java.lang.String path)
```
  - topics
```
public Matrix topics()
```
  - docConcentration
```
public Vector docConcentration()
```
    Description copied from class: LDAModel
    
    Concentration parameter (commonly named "alpha") for the prior placed on documents' distributions over topics ("theta").
    This is the parameter to a Dirichlet distribution.
    
    Specified by:
    
    docConcentration in class LDAModel
    
    Returns:
    (undocumented)
  - topicConcentration
```
public double topicConcentration()
```
    Description copied from class: LDAModel
    
    Concentration parameter (commonly named "beta" or "eta") for the prior placed on topics' distributions over terms.
    This is the parameter to a symmetric Dirichlet distribution.
    Note: The topics' distributions over terms are called "beta" in the original LDA paper by Blei et al., but are called "phi" in many later papers such as Asuncion et al., 2009.
    
    Specified by:
    
    topicConcentration in class LDAModel
    
    Returns:
    (undocumented)
  - gammaShape
```
protected double gammaShape()
```
    Description copied from class: LDAModel
    
    Shape parameter for random initialization of variational parameter gamma. Used for variational inference for perplexity and other test-time computations.
    
    Specified by:
    
    gammaShape in class LDAModel
    
    Returns:
    (undocumented)
  - k
```
public int k()
```
    Description copied from class: LDAModel
    
    Number of topics
    
    Specified by:
    
    k in class LDAModel
  - vocabSize
```
public int vocabSize()
```
    Description copied from class: LDAModel
    
    Vocabulary size (number of terms or terms in the vocabulary)
    
    Specified by:
    
    vocabSize in class LDAModel
  - topicsMatrix
```
public Matrix topicsMatrix()
```
    Description copied from class: LDAModel
    
    Inferred topics, where each topic is represented by a distribution over terms. This is a matrix of size vocabSize x k, where each column is a topic. No guarantees are given about the ordering of the topics.
    
    Specified by:
    
    topicsMatrix in class LDAModel
    
    Returns:
    (undocumented)
  - describeTopics
```
public scala.Tuple2<int[],double[]>[] describeTopics(int maxTermsPerTopic)
```
    Description copied from class: LDAModel
    
    Return the topics described by weighted terms.
    
    Specified by:
    
    describeTopics in class LDAModel
    
    Parameters:
    maxTermsPerTopic - Maximum number of terms to collect for each topic.
    
    Returns:
    Array over topics. Each topic is represented as a pair of matching arrays: (term indices, term weights in topic). Each topic's terms are sorted in order of decreasing weight.
  - formatVersion
```
protected java.lang.String formatVersion()
```
    Description copied from interface: Saveable
    
    Current version of model save/load format.
    
    Specified by:
    
    formatVersion in interface Saveable
  - save
```
public void save(SparkContext sc,
        java.lang.String path)
```
    Description copied from interface: Saveable
    
    Save this model to the given path.
    This saves: - human-readable (JSON) model metadata to path/metadata/ - Parquet formatted data to path/data/
    The model may be loaded using Loader.load.
    
    Specified by:
    
    save in interface Saveable
    
    Parameters:
    sc - Spark context used to save model data.
    path - Path specifying the directory in which to save this model. If the directory already exists, this method throws an exception.
  - logLikelihood
```
public double logLikelihood(RDD<scala.Tuple2<java.lang.Object,Vector>> documents)
```
    Calculates a lower bound on the log likelihood of the entire corpus.
    See Equation (16) in original Online LDA paper.
    
    Parameters:
    documents - test corpus to use for calculating log likelihood
    
    Returns:
    variational lower bound on the log likelihood of the entire corpus
  - logLikelihood
```
public double logLikelihood(JavaPairRDD<java.lang.Long,Vector> documents)
```
    Java-friendly version of logLikelihood
    
    Parameters:
    documents - (undocumented)
    
    Returns:
    (undocumented)
  - logPerplexity
```
public double logPerplexity(RDD<scala.Tuple2<java.lang.Object,Vector>> documents)
```
    Calculate an upper bound bound on perplexity. (Lower is better.) See Equation (16) in original Online LDA paper.
    
    Parameters:
    documents - test corpus to use for calculating perplexity
    
    Returns:
    Variational upper bound on log perplexity per token.
  - logPerplexity
```
public double logPerplexity(JavaPairRDD<java.lang.Long,Vector> documents)
```
    Java-friendly version of logPerplexity
  - topicDistributions
```
public RDD<scala.Tuple2<java.lang.Object,Vector>> topicDistributions(RDD<scala.Tuple2<java.lang.Object,Vector>> documents)
```
    Predicts the topic mixture distribution for each document (often called "theta" in the literature). Returns a vector of zeros for an empty document.
    This uses a variational approximation following Hoffman et al. (2010), where the approximate distribution is called "gamma." Technically, this method returns this approximation "gamma" for each document.
    
    Parameters:
    documents - documents to predict topic mixture distributions for
    
    Returns:
    An RDD of (document ID, topic mixture distribution for document)
  - topicDistributions
```
public JavaPairRDD<java.lang.Long,Vector> topicDistributions(JavaPairRDD<java.lang.Long,Vector> documents)
```
    Java-friendly version of topicDistributions
    
    Parameters:
    documents - (undocumented)
    
    Returns:
    (undocumented)

Class LocalLDAModel

Method Summary

Methods inherited from class org.apache.spark.mllib.clustering.LDAModel

Methods inherited from class java.lang.Object

Method Detail

load

topics

docConcentration

topicConcentration

gammaShape

k

vocabSize

topicsMatrix

describeTopics

formatVersion

save

logLikelihood

logLikelihood

logPerplexity

logPerplexity

topicDistributions

topicDistributions