org.apache.spark.sql.sources

Class OutputWriter

Object

org.apache.spark.sql.sources.OutputWriter

public abstract class OutputWriter
extends Object

::Experimental:: OutputWriter is used together with HadoopFsRelation for persisting rows to the underlying file system. Subclasses of OutputWriter must provide a zero-argument constructor. An OutputWriter instance is created and initialized when a new output file is opened on executor side. This instance is used to persist rows to this single output file.

Since:

1.4.0

Constructor Summary

Constructors

Constructor and Description
OutputWriter()

Method Summary

Methods

Modifier and Type	Method and Description
abstract void	close() Closes the OutputWriter.
abstract void	write(Row row) Persists a single row.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OutputWriter

public OutputWriter()

Method Detail

write

public abstract void write(Row row)

Persists a single row. Invoked on the executor side. When writing to dynamically partitioned tables, dynamic partition columns are not included in rows to be written.

Parameters:

row - (undocumented)

Since:

1.4.0

close

public abstract void close()

Closes the OutputWriter. Invoked on the executor side after all rows are persisted, before the task output is committed.

Since:

1.4.0