org.apache.spark.sql.hive

Class HiveGenericUdtf

Object

org.apache.spark.sql.catalyst.trees.TreeNode<org.apache.spark.sql.catalyst.expressions.Expression>

org.apache.spark.sql.catalyst.expressions.Expression

org.apache.spark.sql.catalyst.expressions.Generator

org.apache.spark.sql.hive.HiveGenericUdtf

All Implemented Interfaces:

java.io.Serializable, HiveInspectors, scala.Equals, scala.Product

public class HiveGenericUdtf
extends org.apache.spark.sql.catalyst.expressions.Generator
implements HiveInspectors, scala.Product, scala.Serializable

Converts a Hive Generic User Defined Table Generating Function (UDTF) to a Generator. Note that the semantics of Generators do not allow Generators to maintain state in between input rows. Thus UDTFs that rely on partitioning dependent operations like calls to close() before producing output will not operate the same as in Hive. However, in practice this should not affect compatibility for most sane UDTFs (e.g. explode or GenericUDTFParseUrlTuple).

Operators that require maintaining state in between input rows should instead be implemented as user defined aggregations, which have clean semantics even in a partitioned execution.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.sql.hive.HiveInspectors

HiveInspectors.typeInfoConversions

Constructor Summary

Constructors

Constructor and Description
HiveGenericUdtf(HiveFunctionWrapper funcWrapper, scala.collection.Seq<String> aliasNames, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> children)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Seq<String>	aliasNames()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression>	children()
scala.collection.TraversableOnce<org.apache.spark.sql.catalyst.expressions.Row>	eval(org.apache.spark.sql.catalyst.expressions.Row input)
HiveFunctionWrapper	funcWrapper()
String	toString()

Methods inherited from class org.apache.spark.sql.catalyst.expressions.Generator

dataType, makeCopy, nullable, output

Methods inherited from class org.apache.spark.sql.catalyst.expressions.Expression

c2, childrenResolved, eval$default$1, f1, f2, foldable, i1, i2, n1, n2, references, resolved

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, collect, fastEquals, flatMap, foreach, generateTreeString, getNodeNumbered, map, mapChildren, nodeName, numberedTreeString, otherCopyArgs, simpleString, stringArgs, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.hive.HiveInspectors

inspectorToDataType, javaClassToDataType, toInspector, toInspector, unwrap, wrap, wrap, wrapperFor

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Constructor Detail

HiveGenericUdtf

public HiveGenericUdtf(HiveFunctionWrapper funcWrapper,
               scala.collection.Seq<String> aliasNames,
               scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> children)

Method Detail

funcWrapper

public HiveFunctionWrapper funcWrapper()

aliasNames

public scala.collection.Seq<String> aliasNames()

children

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> children()

Specified by:

children in class org.apache.spark.sql.catalyst.trees.TreeNode<org.apache.spark.sql.catalyst.expressions.Expression>

eval

public scala.collection.TraversableOnce<org.apache.spark.sql.catalyst.expressions.Row> eval(org.apache.spark.sql.catalyst.expressions.Row input)

Specified by:

eval in class org.apache.spark.sql.catalyst.expressions.Generator

toString

public String toString()

Overrides:

toString in class org.apache.spark.sql.catalyst.trees.TreeNode<org.apache.spark.sql.catalyst.expressions.Expression>