Package org.apache.wayang.spark.channels

Class RddChannel

java.lang.Object

org.apache.wayang.core.plan.executionplan.Channel

org.apache.wayang.spark.channels.RddChannel

public class RddChannel
extends Channel

Describes the situation where one JavaRDD is operated on, producing a further JavaRDD.

NB: We might be more specific: Distinguish between cached/uncached and pipelined/aggregated.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	RddChannel.Instance	ChannelInstance implementation for RddChannels.

Field Summary

Fields

Modifier and Type	Field	Description
static ChannelDescriptor	CACHED_DESCRIPTOR
static ChannelDescriptor	UNCACHED_DESCRIPTOR

Fields inherited from class org.apache.wayang.core.plan.executionplan.Channel

logger

Constructor Summary

Constructors

Constructor	Description
RddChannel(ChannelDescriptor descriptor, OutputSlot<?> outputSlot)

Method Summary

Modifier and Type	Method	Description
RddChannel	copy()	Create a copy of this instance.
RddChannel.Instance	createInstance(Executor executor, OptimizationContext.OperatorContext producerOperatorContext, int producerOutputIndex)	Create a ChannelInstance for this instance.

Methods inherited from class org.apache.wayang.core.plan.executionplan.Channel

addConsumer, addSibling, getCardinalityEstimate, getConsumers, getCorrespondingSlots, getDataSetType, getDescriptor, getOriginal, getProducer, getProducerOperator, getProducerSlot, getSiblings, isBetweenStages, isCopy, isExecutionBreaker, isMarkedForInstrumentation, isReusable, isSuitableForBreakpoint, markForInstrumentation, mergeIntoOriginal, removeSiblings, removeSiblingsWhere, retain, toString, withSiblings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

UNCACHED_DESCRIPTOR

public static final ChannelDescriptor UNCACHED_DESCRIPTOR

CACHED_DESCRIPTOR

public static final ChannelDescriptor CACHED_DESCRIPTOR

Constructor Detail

RddChannel

public RddChannel(ChannelDescriptor descriptor,
                  OutputSlot<?> outputSlot)

Method Detail

copy

public RddChannel copy()

Description copied from class: Channel

Create a copy of this instance. Mimics everything apart from the consumers. Also delimits already executed ExecutionTasks and those that are not executed yet. Be careful when revising this invariant.

Specified by:

copy in class Channel

createInstance

public RddChannel.Instance createInstance(Executor executor,
                                          OptimizationContext.OperatorContext producerOperatorContext,
                                          int producerOutputIndex)

Description copied from class: Channel

Create a ChannelInstance for this instance.

Specified by:

createInstance in class Channel

Parameters:

executor - that manages the resource or null if none