CountVectorizerModel (Spark 2.4.3 JavaDoc)

Object
- org.apache.spark.ml.PipelineStage
- - org.apache.spark.ml.Transformer
  - - org.apache.spark.ml.Model<CountVectorizerModel>
    - - org.apache.spark.ml.feature.CountVectorizerModel

All Implemented Interfaces:

java.io.Serializable, Logging, Params, Identifiable, MLWritable
```
public class CountVectorizerModel
extends Model<CountVectorizerModel>
implements MLWritable
```
Converts a text document to a sparse vector of token counts. param: vocabulary An Array over terms. Only the terms in the vocabulary will be counted.

See Also:

Serialized Form

Constructor Summary

Constructors
Constructor and Description

CountVectorizerModel(String[] vocabulary)

CountVectorizerModel(String uid, String[] vocabulary)

Constructors
Constructor and Description
`CountVectorizerModel(String[] vocabulary)`
`CountVectorizerModel(String uid, String[] vocabulary)`

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`static BooleanParam`	`binary()`
`BooleanParam`	`binary()` Binary toggle to control the output vector values.
`static Params`	`clear(Param<?> param)`
`CountVectorizerModel`	`copy(ParamMap extra)` Creates a copy of this instance with the same UID and some extra params.
`static String`	`explainParam(Param<?> param)`
`static String`	`explainParams()`
`static ParamMap`	`extractParamMap()`
`static ParamMap`	`extractParamMap(ParamMap extra)`
`static <T> scala.Option<T>`	`get(Param<T> param)`
`static boolean`	`getBinary()`
`boolean`	`getBinary()`
`static <T> scala.Option<T>`	`getDefault(Param<T> param)`
`static String`	`getInputCol()`
`static double`	`getMaxDF()`
`double`	`getMaxDF()`
`static double`	`getMinDF()`
`double`	`getMinDF()`
`static double`	`getMinTF()`
`double`	`getMinTF()`
`static <T> T`	`getOrDefault(Param<T> param)`
`static String`	`getOutputCol()`
`static Param<Object>`	`getParam(String paramName)`
`static int`	`getVocabSize()`
`int`	`getVocabSize()`
`static <T> boolean`	`hasDefault(Param<T> param)`
`static boolean`	`hasParam(String paramName)`
`static boolean`	`hasParent()`
`static Param<String>`	`inputCol()`
`static boolean`	`isDefined(Param<?> param)`
`static boolean`	`isSet(Param<?> param)`
`static CountVectorizerModel`	`load(String path)`
`static DoubleParam`	`maxDF()`
`DoubleParam`	`maxDF()` Specifies the maximum number of different documents a term could appear in to be included in the vocabulary.
`static DoubleParam`	`minDF()`
`DoubleParam`	`minDF()` Specifies the minimum number of different documents a term must appear in to be included in the vocabulary.
`static DoubleParam`	`minTF()`
`DoubleParam`	`minTF()` Filter to ignore rare words in a document.
`static Param<String>`	`outputCol()`
`static Param<?>[]`	`params()`
`static void`	`parent_$eq(Estimator<M> x$1)`
`static Estimator<M>`	`parent()`
`static MLReader<CountVectorizerModel>`	`read()`
`static void`	`save(String path)`
`static <T> Params`	`set(Param<T> param, T value)`
`CountVectorizerModel`	`setBinary(boolean value)`
`CountVectorizerModel`	`setInputCol(String value)`
`CountVectorizerModel`	`setMinTF(double value)`
`CountVectorizerModel`	`setOutputCol(String value)`
`static M`	`setParent(Estimator<M> parent)`
`static String`	`toString()`
`Dataset<Row>`	`transform(Dataset<?> dataset)` Transforms the input dataset.
`StructType`	`transformSchema(StructType schema)` :: DeveloperApi ::
`String`	`uid()` An immutable unique ID for the object and its derivatives.
`StructType`	`validateAndTransformSchema(StructType schema)` Validates and transforms the input schema.
`static IntParam`	`vocabSize()`
`IntParam`	`vocabSize()` Max size of the vocabulary.
`String[]`	`vocabulary()`
`MLWriter`	`write()` Returns an `MLWriter` instance for this ML instance.

Methods inherited from class org.apache.spark.ml.Model
hasParent, parent, setParent

Methods inherited from class org.apache.spark.ml.Transformer
transform, transform, transform

Methods inherited from class Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.param.shared.HasInputCol
getInputCol, inputCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasOutputCol
getOutputCol, outputCol

Methods inherited from interface org.apache.spark.ml.param.Params
clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable
toString

Methods inherited from interface org.apache.spark.ml.util.MLWritable
save

Methods inherited from interface org.apache.spark.internal.Logging
initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

- Constructor Detail
  - CountVectorizerModel
```
public CountVectorizerModel(String uid,
                            String[] vocabulary)
```
  - CountVectorizerModel
```
public CountVectorizerModel(String[] vocabulary)
```
- Method Detail
  - read
```
public static MLReader<CountVectorizerModel> read()
```
  - load
```
public static CountVectorizerModel load(String path)
```
  - toString
```
public static String toString()
```
  - params
```
public static Param<?>[] params()
```
  - explainParam
```
public static String explainParam(Param<?> param)
```
  - explainParams
```
public static String explainParams()
```
  - isSet
```
public static final boolean isSet(Param<?> param)
```
  - isDefined
```
public static final boolean isDefined(Param<?> param)
```
  - hasParam
```
public static boolean hasParam(String paramName)
```
  - getParam
```
public static Param<Object> getParam(String paramName)
```
  - set
```
public static final <T> Params set(Param<T> param,
                                   T value)
```
  - get
```
public static final <T> scala.Option<T> get(Param<T> param)
```
  - clear
```
public static final Params clear(Param<?> param)
```
  - getOrDefault
```
public static final <T> T getOrDefault(Param<T> param)
```
  - getDefault
```
public static final <T> scala.Option<T> getDefault(Param<T> param)
```
  - hasDefault
```
public static final <T> boolean hasDefault(Param<T> param)
```
  - extractParamMap
```
public static final ParamMap extractParamMap(ParamMap extra)
```
  - extractParamMap
```
public static final ParamMap extractParamMap()
```
  - parent
```
public static Estimator<M> parent()
```
  - parent_$eq
```
public static void parent_$eq(Estimator<M> x$1)
```
  - setParent
```
public static M setParent(Estimator<M> parent)
```
  - hasParent
```
public static boolean hasParent()
```
  - inputCol
```
public static final Param<String> inputCol()
```
  - getInputCol
```
public static final String getInputCol()
```
  - outputCol
```
public static final Param<String> outputCol()
```
  - getOutputCol
```
public static final String getOutputCol()
```
  - vocabSize
```
public static IntParam vocabSize()
```
  - getVocabSize
```
public static int getVocabSize()
```
  - minDF
```
public static DoubleParam minDF()
```
  - getMinDF
```
public static double getMinDF()
```
  - maxDF
```
public static DoubleParam maxDF()
```
  - getMaxDF
```
public static double getMaxDF()
```
  - minTF
```
public static DoubleParam minTF()
```
  - getMinTF
```
public static double getMinTF()
```
  - binary
```
public static BooleanParam binary()
```
  - getBinary
```
public static boolean getBinary()
```
  - save
```
public static void save(String path)
                 throws java.io.IOException
```
    Throws:
    
    java.io.IOException
  - uid
```
public String uid()
```
    Description copied from interface: Identifiable
    
    An immutable unique ID for the object and its derivatives.
    
    Specified by:
    
    uid in interface Identifiable
    
    Returns:
    
    (undocumented)
  - vocabulary
```
public String[] vocabulary()
```
  - setInputCol
```
public CountVectorizerModel setInputCol(String value)
```
  - setOutputCol
```
public CountVectorizerModel setOutputCol(String value)
```
  - setMinTF
```
public CountVectorizerModel setMinTF(double value)
```
  - setBinary
```
public CountVectorizerModel setBinary(boolean value)
```
  - transform
```
public Dataset<Row> transform(Dataset<?> dataset)
```
    Description copied from class: Transformer
    
    Transforms the input dataset.
    
    Specified by:
    
    transform in class Transformer
    
    Parameters:
    
    dataset - (undocumented)
    
    Returns:
    
    (undocumented)
  - transformSchema
```
public StructType transformSchema(StructType schema)
```
    Description copied from class: PipelineStage
    
    :: DeveloperApi ::
    Check transform validity and derive the output schema from the input schema.
    We check validity for interactions between parameters during transformSchema and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by Param.validate().
    Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
    
    Specified by:
    
    transformSchema in class PipelineStage
    
    Parameters:
    
    schema - (undocumented)
    
    Returns:
    
    (undocumented)
  - copy
```
public CountVectorizerModel copy(ParamMap extra)
```
    Description copied from interface: Params
    
    Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
    
    Specified by:
    
    copy in interface Params
    
    Specified by:
    
    copy in class Model<CountVectorizerModel>
    
    Parameters:
    
    extra - (undocumented)
    
    Returns:
    
    (undocumented)
  - write
```
public MLWriter write()
```
    Description copied from interface: MLWritable
    
    Returns an MLWriter instance for this ML instance.
    
    Specified by:
    
    write in interface MLWritable
    
    Returns:
    
    (undocumented)
  - binary
```
public BooleanParam binary()
```
    Binary toggle to control the output vector values. If True, all nonzero counts (after minTF filter applied) are set to 1. This is useful for discrete probabilistic models that model binary events rather than integer counts. Default: false
    
    Returns:
    
    (undocumented)
  - getBinary
```
public boolean getBinary()
```
  - getMaxDF
```
public double getMaxDF()
```
  - getMinDF
```
public double getMinDF()
```
  - getMinTF
```
public double getMinTF()
```
  - getVocabSize
```
public int getVocabSize()
```
  - maxDF
```
public DoubleParam maxDF()
```
    Specifies the maximum number of different documents a term could appear in to be included in the vocabulary. A term that appears more than the threshold will be ignored. If this is an integer greater than or equal to 1, this specifies the maximum number of documents the term could appear in; if this is a double in [0,1), then this specifies the maximum fraction of documents the term could appear in.
    Default: (2^63^) - 1
    
    Returns:
    
    (undocumented)
  - minDF
```
public DoubleParam minDF()
```
    Specifies the minimum number of different documents a term must appear in to be included in the vocabulary. If this is an integer greater than or equal to 1, this specifies the number of documents the term must appear in; if this is a double in [0,1), then this specifies the fraction of documents.
    Default: 1.0
    
    Returns:
    
    (undocumented)
  - minTF
```
public DoubleParam minTF()
```
    Filter to ignore rare words in a document. For each document, terms with frequency/count less than the given threshold are ignored. If this is an integer greater than or equal to 1, then this specifies a count (of times the term must appear in the document); if this is a double in [0,1), then this specifies a fraction (out of the document's token count).
    Note that the parameter is only used in transform of CountVectorizerModel and does not affect fitting.
    Default: 1.0
    
    Returns:
    
    (undocumented)
  - validateAndTransformSchema
```
public StructType validateAndTransformSchema(StructType schema)
```
    Validates and transforms the input schema.
  - vocabSize
```
public IntParam vocabSize()
```
    Max size of the vocabulary. CountVectorizer will build a vocabulary that only considers the top vocabSize terms ordered by term frequency across the corpus.
    Default: 2^18^
    
    Returns:
    
    (undocumented)

Class CountVectorizerModel

Constructor Summary

Method Summary

Methods inherited from class org.apache.spark.ml.Model

Methods inherited from class org.apache.spark.ml.Transformer

Methods inherited from class Object

Methods inherited from interface org.apache.spark.ml.param.shared.HasInputCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasOutputCol

Methods inherited from interface org.apache.spark.ml.param.Params

Methods inherited from interface org.apache.spark.ml.util.Identifiable

Methods inherited from interface org.apache.spark.ml.util.MLWritable

Methods inherited from interface org.apache.spark.internal.Logging

Constructor Detail

CountVectorizerModel

CountVectorizerModel

Method Detail

read

load

toString

params

explainParam

explainParams

isSet

isDefined

hasParam

getParam

set

get

clear

getOrDefault

getDefault

hasDefault

extractParamMap

extractParamMap

parent

parent_$eq

setParent

hasParent

inputCol

getInputCol

outputCol

getOutputCol

vocabSize

getVocabSize

minDF

getMinDF

maxDF

getMaxDF

minTF

getMinTF

binary

getBinary

save

uid

vocabulary

setInputCol

setOutputCol

setMinTF

setBinary

transform

transformSchema

copy

write

binary

getBinary

getMaxDF

getMinDF

getMinTF

getVocabSize

maxDF

minDF

minTF

validateAndTransformSchema

vocabSize