Package org.apache.wayang.basic.operators

Class MapOperator<InputType,OutputType>

java.lang.Object

org.apache.wayang.core.plan.wayangplan.OperatorBase

org.apache.wayang.core.plan.wayangplan.UnaryToUnaryOperator<InputType,OutputType>

org.apache.wayang.basic.operators.MapOperator<InputType,OutputType>

All Implemented Interfaces:

java.io.Serializable, ActualOperator, ElementaryOperator, Operator

Direct Known Subclasses:

FlinkMapOperator, JavaMapOperator, JdbcProjectionOperator, SparkMapOperator

public class MapOperator<InputType,OutputType>
extends UnaryToUnaryOperator<InputType,OutputType>

A map operator represents semantics as they are known from frameworks, such as Spark and Flink. It pulls each available element from the input slot, applies a function to it, and pushes that element to the output slot.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

OperatorBase.GsonSerializer

Field Summary

Fields

Modifier and Type	Field	Description
protected TransformationDescriptor<InputType,OutputType>	functionDescriptor	Function that this operator applies to the input elements.

Fields inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

inputSlots, outputSlots, STANDARD_OPERATOR_ARGS

Fields inherited from interface org.apache.wayang.core.plan.wayangplan.Operator

FIRST_EPOCH

Constructor Summary

Constructors

Constructor	Description
MapOperator(MapOperator<InputType,OutputType> that)	Copies an instance (exclusive of broadcasts).
MapOperator(FunctionDescriptor.SerializableFunction<InputType,OutputType> function, java.lang.Class<InputType> inputTypeClass, java.lang.Class<OutputType> outputTypeClass)	Creates a new instance.
MapOperator(TransformationDescriptor<InputType,OutputType> functionDescriptor)	Creates a new instance.
MapOperator(TransformationDescriptor<InputType,OutputType> functionDescriptor, DataSetType<InputType> inputType, DataSetType<OutputType> outputType)	Creates a new instance.

Method Summary

Modifier and Type	Method	Description
java.util.Optional<CardinalityEstimator>	createCardinalityEstimator(int outputIndex, Configuration configuration)	Provide a CardinalityEstimator for the OutputSlot at outputIndex.
static <Input,Output> MapOperator<Input,Output>	createProjection(java.lang.Class<Input> inputClass, java.lang.Class<Output> outputClass, java.lang.String... fieldNames)	Creates a new instance that projects the given fields.
static MapOperator<Record,Record>	createProjection(RecordType inputType, java.lang.String... fieldNames)	Creates a new instance that projects the given fields of Records.
TransformationDescriptor<InputType,OutputType>	getFunctionDescriptor()

Methods inherited from class org.apache.wayang.core.plan.wayangplan.UnaryToUnaryOperator

getInput, getInputType, getOutput, getOutputType

Methods inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

accept, addBroadcastInput, addTargetPlatform, at, collectMappedInputSlots, collectMappedOutputSlots, copy, createCopy, getAllInputs, getAllOutputs, getCardinalityEstimator, getContainer, getEpoch, getName, getOriginal, getSimpleClassName, getTargetPlatforms, isAuxiliary, isSupportingBroadcastInputs, propagateInputCardinality, propagateOutputCardinality, setAuxiliary, setCardinalityEstimator, setContainer, setEpoch, setName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.ActualOperator

accept

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.ElementaryOperator

getCardinalityEstimator, isAuxiliary, setAuxiliary, setCardinalityEstimator

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.Operator

addBroadcastInput, addTargetPlatform, broadcastTo, broadcastTo, collectMappedInputSlots, collectMappedOutputSlots, connectTo, connectTo, getAllInputs, getAllOutputs, getCardinalityPusher, getContainer, getEffectiveOccupant, getEffectiveOccupant, getEpoch, getEstimationContextProperties, getForwards, getInnermostLoop, getInput, getInput, getLoopStack, getName, getNumBroadcastInputs, getNumInputs, getNumOutputs, getNumRegularInputs, getOuterInputSlot, getOutermostInputSlot, getOutermostOutputSlots, getOutput, getOutput, getParent, getTargetPlatforms, isAlternative, isElementary, isExecutionOperator, isFeedbackInput, isFeedforwardOutput, isLoopHead, isLoopSubplan, isOwnerOf, isReading, isSink, isSource, isSubplan, isSupportingBroadcastInputs, isUnconnected, propagateInputCardinality, propagateOutputCardinality, propagateOutputCardinality, setContainer, setEpoch, setInput, setName, setOutput

Field Detail

functionDescriptor

protected final TransformationDescriptor<InputType,OutputType> functionDescriptor

Function that this operator applies to the input elements.

Constructor Detail

MapOperator

public MapOperator(FunctionDescriptor.SerializableFunction<InputType,OutputType> function,
                   java.lang.Class<InputType> inputTypeClass,
                   java.lang.Class<OutputType> outputTypeClass)

Creates a new instance.

MapOperator

public MapOperator(TransformationDescriptor<InputType,OutputType> functionDescriptor)

Creates a new instance.

MapOperator

public MapOperator(TransformationDescriptor<InputType,OutputType> functionDescriptor,
                   DataSetType<InputType> inputType,
                   DataSetType<OutputType> outputType)

Creates a new instance.

MapOperator

public MapOperator(MapOperator<InputType,OutputType> that)

Copies an instance (exclusive of broadcasts).

Parameters:

that - that should be copied

Method Detail

createProjection

public static <Input,Output> MapOperator<Input,Output> createProjection(java.lang.Class<Input> inputClass,
                                                                                    java.lang.Class<Output> outputClass,
                                                                                    java.lang.String... fieldNames)

Creates a new instance that projects the given fields.

Parameters:

fieldNames - the field names for the projected fields

Returns:

the new instance

createProjection

public static MapOperator<Record,Record> createProjection(RecordType inputType,
                                                                java.lang.String... fieldNames)

Creates a new instance that projects the given fields of Records.

Parameters:

fieldNames - the field names for the projected fields

Returns:

the new instance

getFunctionDescriptor

public TransformationDescriptor<InputType,OutputType> getFunctionDescriptor()

createCardinalityEstimator

public java.util.Optional<CardinalityEstimator> createCardinalityEstimator(int outputIndex,
                                                                           Configuration configuration)

Description copied from interface: ElementaryOperator

Provide a CardinalityEstimator for the OutputSlot at outputIndex.

Parameters:

outputIndex - index of the OutputSlot for that the CardinalityEstimator is requested

configuration - if the CardinalityEstimator depends on further ones, use this to obtain the latter

Returns:

an Optional that might provide the requested instance