Adding new operators in Wayang

This guide shows the 3 steps that developers need to follow if they want to add new operators in Wayang. We use the Map operator as an example.

Step 1: Add a Wayang operator

Wayang operators are located under the wayang-basic in the org.apache.wayang.basic.operators package.
An operator needs to extend from one of the following abstract classes: UnaryToUnaryOperator, BinaryToUnaryOperator, UnarySource, UnarySink.
For a unary to unary operator, see for example here.

For enhanced performance in Wayang, consider adding a cardinality estimator by overriding the createCardinalityEstimator() function as here.

Step 2: Add the (platform-specific) execution operators

Execution operators are located under the corresponding module of wayang-platforms. For instance, Java execution operators are located in the org.apache.wayang.java.operators package of the wayang-java module.
An execution operator needs to extend from its corresponding Wayang operator and implement the corresponding platform operator interface.
For the above MapOperator, the following is the corresponding JavaMapOperator.

For enhanced performance in Wayang, consider adding a load function as well:
For this you need to overwrite the getLoadProfileEstimatorConfigurationKey() function and provide the right key that will then be read from a properties file. For the JavaMapOperator it's: wayang.java.map.load. Then add in the corresponding properties file (e.g., this is for the java executor) the template which is the mathematical formula that represents the cost of this operator and an instantiation of it. See here for the example of the map operator.

Step 3: Add mappings

Create mappings from the Wayang operator to the platform-specific execution operators.
The mappings are located in the corresponding execution module in the org.apache.wayang.java.operators package.
For the above MapOperator and JavaMapOperator, see here.

After that you need to declare this mapping in Wayang in the corresponding Mappings class.