org.apache.spark.sql.execution

Class BatchPythonEvaluation

Object

org.apache.spark.sql.catalyst.trees.TreeNode<PlanType>

org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

org.apache.spark.sql.execution.SparkPlan

org.apache.spark.sql.execution.BatchPythonEvaluation

All Implemented Interfaces:

java.io.Serializable, Logging, scala.Equals, scala.Product

public class BatchPythonEvaluation
extends SparkPlan
implements scala.Product, scala.Serializable

:: DeveloperApi :: Uses PythonRDD to evaluate a PythonUDF, one partition of tuples at a time. The input data is cached and zipped with the result of the udf evaluation.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
BatchPythonEvaluation(PythonUDF udf, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output, SparkPlan child)

Method Summary

Methods

Modifier and Type	Method and Description
SparkPlan	child()
scala.collection.immutable.List<SparkPlan>	children()
RDD<org.apache.spark.sql.catalyst.expressions.Row>	execute() Runs this query returning the result as an RDD.
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	output()
PythonUDF	udf()

Methods inherited from class org.apache.spark.sql.execution.SparkPlan

codegenEnabled, executeCollect, makeCopy, outputPartitioning, requiredChildDistribution

Methods inherited from class org.apache.spark.sql.catalyst.plans.QueryPlan

expressions, inputSet, missingInput, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionUp$1, outputSet, printSchema, references, schema, schemaString, simpleString, statePrefix, transformAllExpressions, transformExpressions, transformExpressionsDown, transformExpressionsUp

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, collect, fastEquals, flatMap, foreach, generateTreeString, getNodeNumbered, map, mapChildren, nodeName, numberedTreeString, otherCopyArgs, stringArgs, toString, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

BatchPythonEvaluation

public BatchPythonEvaluation(PythonUDF udf,
                     scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output,
                     SparkPlan child)

Method Detail

udf

public PythonUDF udf()

output

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output()

Specified by:

output in class org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

child

public SparkPlan child()

children

public scala.collection.immutable.List<SparkPlan> children()

Specified by:

children in class org.apache.spark.sql.catalyst.trees.TreeNode<SparkPlan>

execute

public RDD<org.apache.spark.sql.catalyst.expressions.Row> execute()

Description copied from class: SparkPlan

Runs this query returning the result as an RDD.

Specified by:

execute in class SparkPlan