pyspark.pandas.groupby.GroupBy.fillna#

GroupBy.fillna(value=None, method=None, axis=None, inplace=False, limit=None)[source]#

Fill NA/NaN values in group.

Parameters

valuescalar, dict, Series: Value to use to fill holes. alternately a dict/Series of values specifying which value to use for each column. DataFrame is not supported.
method{‘backfill’, ‘bfill’, ‘pad’, ‘ffill’, None}, default None: Method to use for filling holes in reindexed Series pad / ffill: propagate last valid observation forward to next valid backfill / bfill: use NEXT valid observation to fill gap

Deprecated since version 4.0.0.
axis{0 or index}: 1 and columns are not supported.

Deprecated since version 4.0.0: For axis=1, operate on the underlying object instead. Otherwise the axis keyword is not necessary.
inplaceboolean, default False: Fill in place (do not create a new object)
limitint, default None: If method is specified, this is the maximum number of consecutive NaN values to forward/backward fill. In other words, if there is a gap with more than this number of consecutive NaNs, it will only be partially filled. If method is not specified, this is the maximum number of entries along the entire axis where NaNs will be filled. Must be greater than 0 if not None

Deprecated since version 4.0.0.

Returns

DataFrame: DataFrame with NA entries filled.

Examples

>>> df = ps.DataFrame({
...     'A': [1, 1, 2, 2],
...     'B': [2, 4, None, 3],
...     'C': [None, None, None, 1],
...     'D': [0, 1, 5, 4]
...     },
...     columns=['A', 'B', 'C', 'D'])
>>> df
   A    B    C  D
0  1  2.0  NaN  0
1  1  4.0  NaN  1
2  2  NaN  NaN  5
3  2  3.0  1.0  4

We can also propagate non-null values forward or backward in group.

>>> df.groupby(['A'])['B'].fillna(method='ffill').sort_index()
0    2.0
1    4.0
2    NaN
3    3.0
Name: B, dtype: float64

>>> df.groupby(['A']).fillna(method='bfill').sort_index()
     B    C  D
0  2.0  NaN  0
1  4.0  NaN  1
2  3.0  1.0  5
3  3.0  1.0  4