pyspark.sql.DataFrame.foreachPartition

DataFrame.foreachPartition(f: Callable[[Iterator[pyspark.sql.types.Row]], None]) → None

Applies the f function to each partition of this DataFrame.

This a shorthand for df.rdd.foreachPartition().

New in version 1.3.0.

Examples

>>> def f(people):
...     for person in people:
...         print(person.name)
>>> df.foreachPartition(f)

pyspark.sql.DataFrame.foreach pyspark.sql.DataFrame.freqItems