Package org.apache.wayang.spark.channels

Class DatasetChannel

java.lang.Object

org.apache.wayang.core.plan.executionplan.Channel

org.apache.wayang.spark.channels.DatasetChannel

public class DatasetChannel
extends Channel

Channel implementation that transports Spark Datasets (i.e., DataFrames).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	DatasetChannel.Instance	ChannelInstance for DatasetChannels.

Field Summary

Fields

Modifier and Type	Field	Description
static final ChannelDescriptor	CACHED_DESCRIPTOR
static final ChannelDescriptor	UNCACHED_DESCRIPTOR

Fields inherited from class org.apache.wayang.core.plan.executionplan.Channel

logger

Constructor Summary

Constructors

Constructor	Description
DatasetChannel(ChannelDescriptor descriptor, OutputSlot<?> outputSlot)

Method Summary

Modifier and Type	Method	Description
DatasetChannel	copy()	Create a copy of this instance.
DatasetChannel.Instance	createInstance(Executor executor, OptimizationContext.OperatorContext producerOperatorContext, int producerOutputIndex)	Create a ChannelInstance for this instance.

Methods inherited from class org.apache.wayang.core.plan.executionplan.Channel

addConsumer, addSibling, getCardinalityEstimate, getConsumers, getCorrespondingSlots, getDataSetType, getDescriptor, getOriginal, getProducer, getProducerOperator, getProducerSlot, getSiblings, isBetweenStages, isCopy, isExecutionBreaker, isMarkedForInstrumentation, isReusable, isSuitableForBreakpoint, markForInstrumentation, mergeIntoOriginal, removeSiblings, removeSiblingsWhere, retain, toString, withSiblings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

UNCACHED_DESCRIPTOR

public static final ChannelDescriptor UNCACHED_DESCRIPTOR

CACHED_DESCRIPTOR

public static final ChannelDescriptor CACHED_DESCRIPTOR

Constructor Details

DatasetChannel

public DatasetChannel(ChannelDescriptor descriptor,
 OutputSlot<?> outputSlot)

Method Details

copy

public DatasetChannel copy()

Description copied from class: Channel

Create a copy of this instance. Mimics everything apart from the consumers. Also delimits already executed ExecutionTasks and those that are not executed yet. Be careful when revising this invariant.

Specified by:

copy in class Channel

createInstance

public DatasetChannel.Instance createInstance(Executor executor,
 OptimizationContext.OperatorContext producerOperatorContext,
 int producerOutputIndex)

Description copied from class: Channel

Create a ChannelInstance for this instance.

Specified by:

createInstance in class Channel

Parameters:

executor - that manages the resource or null if none