pyspark.pandas.groupby.DataFrameGroupBy.cov#

DataFrameGroupBy.cov(min_periods=None, ddof=1, numeric_only=False)[source]#

Compute pairwise covariance of columns, excluding NA/null values.

The returned DataFrame is the covariance matrix of the columns of the DataFrame within each group.

Both NA and null values are automatically excluded from the calculation. A threshold can be set for the minimum number of observations for each value created. Comparisons with observations below this threshold will be returned as NaN.

Added in version 4.3.0.

Parameters:

min_periodsint, optional: Minimum number of observations required per pair of columns to have a valid result.
ddofint, default 1: Delta degrees of freedom. The divisor used in calculations is N - ddof, where N represents the number of elements.
numeric_onlybool, default False: Include only float, int or boolean data.

Returns:

DataFrame: The covariance matrix of the series of the DataFrame within each group.

See also

DataFrame.cov
Series.cov

Examples

>>> df = ps.DataFrame(
...     {"A": [1, 1, 2, 2, 2], "B": [1, 2, 3, 4, 5], "C": [4, 6, 7, 9, 11]},
...     columns=["A", "B", "C"])
>>> df.groupby("A").cov().sort_index()
        B    C
A
1 B  0.5  1.0
  C  1.0  2.0
2 B  1.0  2.0
  C  2.0  4.0

>>> df.groupby("A").cov(ddof=0).sort_index()
            B         C
A
1 B  0.250000  0.500000
  C  0.500000  1.000000
2 B  0.666667  1.333333
  C  1.333333  2.666667

>>> df.groupby("A").cov(min_periods=3).sort_index()
        B    C
A
1 B  NaN  NaN
  C  NaN  NaN
2 B  1.0  2.0
  C  2.0  4.0