pyspark.sql.functions.try_variant_get

pyspark.sql.functions.try_variant_get(v, path, targetType)

Extracts a sub-variant from v according to path, and then cast the sub-variant to targetType. Returns null if the path does not exist or the cast fails.

Added in version 4.0.0.

Parameters:

vColumn or str

a variant column or column name A column that evaluates to a variant.

pathColumn or str

a column containing the extraction path strings or a string representing the extraction path. A valid path should start with $ and is followed by zero or more segments like A column that evaluates to a string. [123], .name, [‘name’], or [“name”].

targetTypestr

A DDL-formatted type string. Must be a constant. the target data type to cast into, in a DDL-formatted string

Returns:

Column

a column of targetType representing the extracted result Returns a column of the type given by targetType.

Examples

>>> df = spark.createDataFrame([ {'json': '''{ "a" : 1 }''', 'path': '$.a'} ])
>>> df.select(try_variant_get(parse_json(df.json), "$.a", "int").alias("r")).collect()
[Row(r=1)]
>>> df.select(try_variant_get(parse_json(df.json), "$.b", "int").alias("r")).collect()
[Row(r=None)]
>>> df.select(try_variant_get(parse_json(df.json), "$.a", "binary").alias("r")).collect()
[Row(r=None)]
>>> df.select(try_variant_get(parse_json(df.json), df.path, "int").alias("r")).collect()
[Row(r=1)]