pyspark.sql.functions.date_diff

pyspark.sql.functions.date_diff(end, start)

Returns the number of days from start to end.

Added in version 3.5.0.

Parameters:

endColumn or column name

to date column to work on. A column that evaluates to a date.

startColumn or column name

from date column to work on. A column that evaluates to a date.

Returns:

Column

difference in days between two dates. Returns a column that evaluates to an integer.

See also

pyspark.sql.functions.dateadd()

pyspark.sql.functions.date_add()

pyspark.sql.functions.date_sub()

pyspark.sql.functions.datediff()

pyspark.sql.functions.timestamp_diff()

pyspark.sql.functions.time_diff()

Examples

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('2015-04-08','2015-05-10')], ['d1', 'd2'])
>>> df.select('*', sf.date_diff('d1', 'd2')).show()
+----------+----------+-----------------+
|        d1|        d2|date_diff(d1, d2)|
+----------+----------+-----------------+
|2015-04-08|2015-05-10|              -32|
+----------+----------+-----------------+

>>> df.select('*', sf.date_diff(df.d2, df.d1)).show()
+----------+----------+-----------------+
|        d1|        d2|date_diff(d2, d1)|
+----------+----------+-----------------+
|2015-04-08|2015-05-10|               32|
+----------+----------+-----------------+