pyspark.sql.functions.input_file_block_length

pyspark.sql.functions.input_file_block_length()

Returns the length of the block being read, or -1 if not available.

New in version 3.5.0.

Examples

>>> df = spark.read.text("python/test_support/sql/ages_newlines.csv", lineSep=",")
>>> df.select(input_file_block_length().alias('r')).first()
Row(r=87)