Package org.apache.wayang.basic.operators

Class CoGroupOperator<InputType0,InputType1,Key>

java.lang.Object

org.apache.wayang.core.plan.wayangplan.OperatorBase

org.apache.wayang.core.plan.wayangplan.BinaryToUnaryOperator<InputType0,InputType1,Tuple2<Iterable<InputType0>,Iterable<InputType1>>>

org.apache.wayang.basic.operators.CoGroupOperator<InputType0,InputType1,Key>

All Implemented Interfaces:

Serializable, ActualOperator, ElementaryOperator, Operator

Direct Known Subclasses:

FlinkCoGroupOperator, JavaCoGroupOperator, SparkCoGroupOperator

public class CoGroupOperator<InputType0,InputType1,Key>
extends BinaryToUnaryOperator<InputType0,InputType1,Tuple2<Iterable<InputType0>,Iterable<InputType1>>>

This operator groups both inputs by some key and then matches groups with the same key. If a key appears in only one of the input datasets, then the according group is matched with an empty group.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

OperatorBase.GsonSerializer

Field Summary

Fields

Modifier and Type	Field	Description
protected final TransformationDescriptor<InputType0,Key>	keyDescriptor0
protected final TransformationDescriptor<InputType1,Key>	keyDescriptor1

Fields inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

inputSlots, outputSlots, STANDARD_OPERATOR_ARGS

Fields inherited from interface org.apache.wayang.core.plan.wayangplan.Operator

FIRST_EPOCH

Constructor Summary

Constructors

Constructor	Description
CoGroupOperator(CoGroupOperator<InputType0,InputType1,Key> that)	Copies an instance (exclusive of broadcasts).
CoGroupOperator(FunctionDescriptor.SerializableFunction<InputType0,Key> keyExtractor0, FunctionDescriptor.SerializableFunction<InputType1,Key> keyExtractor1, Class<InputType0> input0Class, Class<InputType1> input1Class, Class<Key> keyClass)
CoGroupOperator(TransformationDescriptor<InputType0,Key> keyDescriptor0, TransformationDescriptor<InputType1,Key> keyDescriptor1)
CoGroupOperator(TransformationDescriptor<InputType0,Key> keyDescriptor0, TransformationDescriptor<InputType1,Key> keyDescriptor1, DataSetType<InputType0> inputType0, DataSetType<InputType1> inputType1)

Method Summary

Modifier and Type	Method	Description
Optional<CardinalityEstimator>	createCardinalityEstimator(int outputIndex, Configuration configuration)	Provide a CardinalityEstimator for the OutputSlot at outputIndex.
TransformationDescriptor<InputType0,Key>	getKeyDescriptor0()
TransformationDescriptor<InputType1,Key>	getKeyDescriptor1()

Methods inherited from class org.apache.wayang.core.plan.wayangplan.BinaryToUnaryOperator

getInputType0, getInputType1, getOutputType

Methods inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

accept, addBroadcastInput, addTargetPlatform, at, collectMappedInputSlots, collectMappedOutputSlots, copy, createCopy, getAllInputs, getAllOutputs, getCardinalityEstimator, getContainer, getEpoch, getName, getOriginal, getSimpleClassName, getTargetPlatforms, isAuxiliary, isSupportingBroadcastInputs, propagateInputCardinality, propagateOutputCardinality, setAuxiliary, setCardinalityEstimator, setContainer, setEpoch, setName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.ActualOperator

accept

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.ElementaryOperator

getCardinalityEstimator, isAuxiliary, setAuxiliary, setCardinalityEstimator

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.Operator

addBroadcastInput, addTargetPlatform, broadcastTo, broadcastTo, collectMappedInputSlots, collectMappedOutputSlots, connectTo, connectTo, getAllInputs, getAllOutputs, getCardinalityPusher, getContainer, getEffectiveOccupant, getEffectiveOccupant, getEpoch, getEstimationContextProperties, getForwards, getInnermostLoop, getInput, getInput, getLoopStack, getName, getNumBroadcastInputs, getNumInputs, getNumOutputs, getNumRegularInputs, getOuterInputSlot, getOutermostInputSlot, getOutermostOutputSlots, getOutput, getOutput, getParent, getTargetPlatforms, isAlternative, isConversion, isElementary, isExecutionOperator, isFeedbackInput, isFeedforwardOutput, isLoopHead, isLoopSubplan, isOwnerOf, isReading, isSink, isSource, isSubplan, isSupportingBroadcastInputs, isUnconnected, propagateInputCardinality, propagateOutputCardinality, propagateOutputCardinality, replaceWith, setContainer, setEpoch, setInput, setName, setOutput

Field Details

keyDescriptor0

protected final TransformationDescriptor<InputType0,Key> keyDescriptor0

keyDescriptor1

protected final TransformationDescriptor<InputType1,Key> keyDescriptor1

Constructor Details

CoGroupOperator

public CoGroupOperator(FunctionDescriptor.SerializableFunction<InputType0,Key> keyExtractor0,
 FunctionDescriptor.SerializableFunction<InputType1,Key> keyExtractor1,
 Class<InputType0> input0Class,
 Class<InputType1> input1Class,
 Class<Key> keyClass)

CoGroupOperator

public CoGroupOperator(TransformationDescriptor<InputType0,Key> keyDescriptor0,
 TransformationDescriptor<InputType1,Key> keyDescriptor1)

CoGroupOperator

public CoGroupOperator(TransformationDescriptor<InputType0,Key> keyDescriptor0,
 TransformationDescriptor<InputType1,Key> keyDescriptor1,
 DataSetType<InputType0> inputType0,
 DataSetType<InputType1> inputType1)

CoGroupOperator

public CoGroupOperator(CoGroupOperator<InputType0,InputType1,Key> that)

Copies an instance (exclusive of broadcasts).

Parameters:

that - that should be copied

Method Details

getKeyDescriptor0

public TransformationDescriptor<InputType0,Key> getKeyDescriptor0()

getKeyDescriptor1

public TransformationDescriptor<InputType1,Key> getKeyDescriptor1()

createCardinalityEstimator

public Optional<CardinalityEstimator> createCardinalityEstimator(int outputIndex,
 Configuration configuration)

Description copied from interface: ElementaryOperator

Provide a CardinalityEstimator for the OutputSlot at outputIndex.

Parameters:

outputIndex - index of the OutputSlot for that the CardinalityEstimator is requested

configuration - if the CardinalityEstimator depends on further ones, use this to obtain the latter

Returns:

an Optional that might provide the requested instance