All Classes and Interfaces
Class
Description
Abstract
CardinalityPusher implementation for OperatorAlternatives and subclasses.Template for
ChannelInstance implementations.Implements a template for
ReferenceCountable objects.AbstractTopologicalTraversal<ActivatorType extends AbstractTopologicalTraversal.Activator<ActivationType>,ActivationType extends AbstractTopologicalTraversal.Activation<ActivatorType>>
Encapsulates logic to traverse a
WayangPlan in a topological, bottom-up manner.Declares that the current traversal should be aborted.
AbstractTopologicalTraversal.Activation<TActivator extends AbstractTopologicalTraversal.Activator<? extends AbstractTopologicalTraversal.Activation<TActivator>>>
Describes a reference to an input of an
AbstractTopologicalTraversal.Activator.AbstractTopologicalTraversal.Activator<TActivation extends AbstractTopologicalTraversal.Activation<? extends AbstractTopologicalTraversal.Activator<TActivation>>>
Wraps a
CardinalityEstimator, thereby caching its input CardinalityEstimates and keeping track
of its dependent CardinalityEstimators.This interface represents any piece of code that takes no input and produces no output but may fail.
Utilities to perform actions.
Operator free of ambiguities.
Created by khayyzy on 5/28/16.
This
OptimizationContext implementation aggregates several OptimizationContexts and exposes
their OptimizationContext.OperatorContext in an aggregated manner.CardinalityEstimator implementation that can have multiple ways of calculating a CardinalityEstimate.This descriptor pertains to functions that take multiple data units and aggregate them into a single data unit.
This source reads a blob file stored in Amazon s3 and outputs the lines as data units.
Mapping from
AmazonS3Source to JavaAmazonS3Source.This interface represents an application metrics in a Spark cluster.
The
ApplicationEnd class represents an application that has ended.The ApplicationStart class implements the Application and SerializableObject interfaces.
An atomic execution describes the smallest work unit considered by Wayang's cost model.
This
JsonSerializer stores the given instances via their Configuration
key, if any, or else by the LoadProfile that they estimate.JsonSerializer implementation for AtomicExecutionGroups.This source reads a blob file stored in Azure Blob Storage and outputs the lines as data units.
Mapping from
AzureBlobStorageSource to JavaAzureBlobStorageSource.A basic data unit type is elementary and not constructed from other data unit types.
Implementation of a battle between to elements.
An operation
Expression.Wraps a
BinaryOperator as a Function2.SparkOperatorProfiler implementation for SparkExecutionOperators with two inputs and one output.This operator has two inputs and a single output.
A mutable bit-mask.
Created by khayyzy on 5/28/16.
Created by khayyzy on 5/28/16.
Describes when to interrupt the execution of an
ExecutionPlan.Channel that represents a broadcasted value.build2ListObject<TypeXPivot extends Comparable<TypeXPivot>,TypeXRef extends Comparable<TypeXRef>,Input extends Copyable<Input>>
Created by khayyzy on 5/28/16.
This utility maintains canonical sets of objects.
An estimate of cardinality within a
WayangPlan expressed as a ProbabilisticIntervalEstimate.CardinalityEstimator that subsumes a DAG of operators, each one providing a local CardinalityEstimator.Computes
CardinalityEstimates.Handles the
CardinalityEstimates of a WayangPlan.Stores cardinalities that have been collected by the
CrossPlatformExecutor.Mapping from
CartesianOperator to FlinkCartesianOperator.Mapping from
CartesianOperator to JavaCartesianOperator.Mapping from
CartesianOperator to SparkCartesianOperator.This operator returns the cartesian product of elements of input datasets.
Models the data movement between to
ExecutionTasks.This graph contains a set of
ChannelConversions.Prefers
ChannelConversionGraph.Trees with lower cost.Prefers
ChannelConversionGraph.Trees with lower cost.ChannelConversions used by the FlinkPlatform.ChannelConversions for the GiraphPlatform.ChannelConversions for the JavaPlatform.Register for the
ChannelConversions supported for this platform.ChannelConversions used by the JavaPlatform.Register for the
ChannelConversions supported for this platform.ChannelConversions for the TensorflowPlatform.Describes a certain
Channel type including further parameters.Represents the actual, allocated resource represented by
Channel.Encapsulates a
ChannelInstance in the lazy execution lineage.Mapping from
CoGroupOperator to FlinkCoGroupOperator.Mapping from
JoinOperator to JavaJoinOperator.Mapping from
CoGroupOperator to SparkCoGroupOperator.This operator groups both inputs by some key and then matches groups with the same key.
Provides a
Collection of objects.This source takes as input a Java
Collection.Mapping from
CollectionSource to FlinkCollectionSource.Mapping from
CollectionSource to JavaCollectionSource.Mapping from
CollectionSource to SparkCollectionSource.Mapping from
CollectionSource to TensorflowCollectionSource.Expression implementation that represents a function with a static implementation.ExecutionResource that itself contains ExecutionResources.A composite operator can be decomposed into smaller operators.
Created by zoi on 22/1/15.
Describes both the configuration of a
WayangContext and Jobs.Breakpoint implementation that disrupts execution if all aggregated Breakpoints request a disruption.A constant
Expression.LoadProfileEstimator that estimates a predefined LoadProfile.Used by
Configurations to provide some value.Created by bertty on 13-07-17.
Utility to expose interfaces that accept a callback as an
Iterator.Provides contextual variables and functions in order to evaluate an
Expression.Implementing objects must be able to provide copies of themselves.
This measurement captures execution costs w.r.t. to Wayang's cost model.
This utility helps to count elements.
Mapping from
CountOperator to FlinkCountOperator.Mapping from
CountOperator to JavaCountOperator.Mapping from
CountOperator to SparkCountOperator.This operator returns the count of elements in this stream.
Executes a (cross-platform)
ExecutionPlan.Iterates all combinations, i.e., the Cartesian product, of given
Iterables.Based on Calcite's CSV enumerator.
Created by khayyzy on 5/28/16.
Created by khayyzy on 5/28/16.
This class describes a database.
Created by khayyzy on 5/28/16.
Utility to create common data generators.
Describes the situation where one
DataSet is operated on, producing a further DataSet.A data set is an abstraction of the Wayang programming model.
A grouped data unit type describes just the structure of data units within a grouped dataset.
The type of data types designate the kind of data that are being passed between operators.
Mapping from
DecisionTreeRegressionOperator to SparkDecisionTreeRegressionOperator.Default implementation of the
CardinalityEstimator.Default
CardinalityPusher implementation.Default implementation of the
ChannelConversion.Default
Context implementation that can be configured.Implementation of
LoadEstimator that uses a single-point cost function.This implementation of
OptimizationContext represents a direct mapping from OptimizationContext.OperatorContext
to executions of the respective Operators.Profiles the reading and writing speed to some directory.
Mapping from
DistinctOperator to FlinkDistinctOperator.Mapping from
DistinctOperator to JavaDistinctOperator.Mapping from
DistinctOperator to SparkDistinctOperator.This operator returns the distinct elements in this dataset.
Mapping from
DLTrainingOperator to TensorflowDLTrainingOperator.Mapping from
DoWhileOperator to FlinkDoWhileOperator.Mapping from
LoopOperator to JavaLoopOperator.Mapping from
LoopOperator to SparkLoopOperator.This operator has three inputs and two outputs.
Create a that remove the elements null
Class create a that genereta only null as convertion
EstimationContext implementation for DynamicLoadEstimators.Adjustable
LoadProfileEstimator implementation.Adjustable
LoadProfileEstimator implementation.Utility to create
DynamicLoadProfileEstimators.This
Plugin can be arbitrarily customized.An enumeration alternative is embedded within an
EnumerationBranch.An enumeration branch is the basic unit for enumeration, i.e., translation from a
WayangPlan to an
ExecutionPlan.EstimatableCost defines an interface for optimizer cost in terms of
methods to estimate a PlanImplementation's cost.
Provides parameters required by
LoadProfileEstimators.Marks a property of an
Operator that is relevant to the estimation process, i.e., should be provided
in EstimationContexts.This exception signals a failed
Expression evaluation.While a function is executed on a certain
Platform, allows access to some information of the context in
which the function is being executed.Encapsulates
AtomicExecutions with a common OptimizationContext.OperatorContext in a lazy execution lineage.Stores execution data have been collected by the
CrossPlatformExecutor.An execution operator is handled by a certain platform.
Represents an executable, cross-platform data flow.
This
Measurement encapsulates an ExecutionPlan.Encapsulates a
Channel of the ExecutionPlan.A directed link between an
ExecutionPlanMeasurement.OperatorNode and a ExecutionPlanMeasurement.ChannelNode (in any order).Encapsulates a
Channel of the ExecutionPlan.Represents a resource allocated for execution that needs to be released manually.
Implements various functionalities of an
ExecutionResource.Resides within a
PlatformExecution and represents the minimum execution unit that is controlled by Wayang.This class models the execution equivalent of
LoopSubplans.Contains a state of the execution of an
ExecutionTaskFlow.Serves as an adapter to include
ExecutionOperators, which are usually parts of WayangPlans, in
ExecutionPlans.Graph of
ExecutionTasks and Channels.Creates an
ExecutionTaskFlow from a PlanImplementation.Propagates a
Channel to its consumers.An executor executes
ExecutionOperators.The Executor interface represents an executor in a Spark cluster.
Factory for
Executors.The ExecutorAdded class represents an executor added event in a distributed computing system.
An event class representing the removal of an executor.
Implements the
ExecutionResource handling as defined by Executor.An implementation of the Executor interface that represents an updated executor.
An experiment comprises
Measurements from one specific Subject execution.CollectionProvider implementation based on a blacklist and a whitelist.A mathematical expression that can be evaluated.
This utility builds
Expressions from an input String.Implements a
Function2 that calls ExtendedFunction.open(ExecutionContext)
of its implementation before delegating the very first Function2.call(Object, Object).Implements a
FlatMapFunction that calls ExtendedFunction.open(ExecutionContext)
of its implementation before delegating the very first Function.call(Object).Used to enrich regular functions with additional life-cycle methods of
Operators.Implements a
Function that calls ExtendedFunction.open(ExecutionContext)
of its implementation before delegating the very first Function.call(Object).Implements a
Function that calls ExtendedFunction.open(ExecutionContext)
of its implementation before delegating the very first Function.call(Object).Wraps a
Function as a FlatMapFunction.Implements a
Function that calls ExtendedFunction.open(ExecutionContext)
of its implementation before delegating the very first Function.call(Object).Created by khayyzy on 5/28/16.
Created by khayyzy on 5/28/16.
Assumes with a confidence of 50% that the output cardinality will be somewhere between 1 and the product of
all 10*input estimates.
Represents a
Channel that is realized via a file/set of files.ChannelDescriptor for FileChannels.Utilities for
FileChannels.Abstraction for accessing a file system.
Tool to work with
FileSystems.Mapping from
FilterOperator to FlinkFilterOperator.Mapping from
FilterOperator to GenericJdbcFilterOperator.Mapping from
FilterOperator to JavaFilterOperator.Mapping from
FilterOperator to PostgresFilterOperator.Mapping from
FilterOperator to SparkFilterOperator.Mapping from
FilterOperator to Sqlite3FilterOperator.This operator returns a new dataset after filtering by applying predicateDescriptor.
Created by khayyzy on 5/28/16.
Describes when to interrupt the execution of an
ExecutionPlan.CardinalityEstimator implementation for Operators with a fix-sized output.This descriptor pertains to functions that consume a single data unit and output a group of data units.
Wraps a
Function as a FlatMapFunction.Mapping from
FlatMapOperator to FlinkFlatMapOperator.Mapping from
FlatMapOperator to JavaFlatMapOperator.Mapping from
FlatMapOperator to SparkFlatMapOperator.A flatmap operator represents semantics as they are known from frameworks, such as Spark and Flink.
Register for relevant components of this module.
Flink implementation of the
CartesianOperator.Wrapper of of Flink for use in Wayang
Flink implementation of the
CoGroupOperator.Converts
DataSetChannel into a CollectionChannelThis is execution operator implements the
CollectionSource.Wraps and manages a Flink
ExecutionEnvironment to avoid steady re-creation.Flink implementation of the
CountOperator.Flink implementation of the
DistinctOperator.Flink implementation of the
DoWhileOperator.ExecutionContext implementation for the FlinkPlatform.Execution operator for the Flink platform.
Executor implementation for the FlinkPlatform.Flink implementation of the
FilterOperator.Flink implementation of the
FlatMapOperator.Flink implementation of the
GlobalMaterializedGroupOperator.Flink implementation of the
GlobalReduceOperator.Flink implementation of the
GroupByOperator.Flink implementation of the
IntersectOperator.Flink implementation of the
JoinOperator.Implementation of the
LocalCallbackSink operator for the Flink platform.Flink implementation of the
RepeatOperator.Flink implementation of the
MapOperator.Flink implementation of the
MapPartitionsOperator.Flink implementation of the
MaterializedGroupByOperator.Operator for the FlinkPlatform that creates a sequence file.Operator for the FlinkPlatform that creates a sequence file.Flink implementation of the
PageRankOperator.A reduce function that takes a sequence of edges and builds the adjacency list for the vertex where the edges
originate.
The function that applies the page rank dampening formula.
Filter that filters vertices where the rank difference is below a threshold.
Join function that distributes a fraction of a vertex's rank to all neighbors.
A map function that assigns an initial rank to all pages.
Platform for Apache Flink.Flink implementation of the
ReduceByOperator.Flink implementation of the
RepeatOperator.Flink implementation of the
RepeatOperator.Flink implementation of the
SampleOperator.Flink implementation of the
SortOperator.Implementation of the
TextFileSink operator for the Flink platform.Provides a
Collection to a Flink job.Created by bertty on 31-10-17.
Flink implementation of the
UnionAllOperator.Flink implementation of the
MapOperator.Formats different general purpose objects.
Instruments only outbound
Channels.CollectionProvider implementation based on a blacklist and a whitelist.Implementation of
KeyValueProvider that uses a Function to provide a value.Used by
Configurations to provide some value.A compiler translates Wayang functions into executable Flink functions.
A compiler translates Wayang functions into executable Java functions.
Compiles
FunctionDescriptors to SQL clauses.A compiler translates Wayang functions into executable Java functions.
Spark function for building pair RDDs.
Spark function for aggregating data quanta.
Describes functions coming from Wayang, designated for Spark.
A function operates on single data units or collections of those.
Extends a
FunctionDescriptor.SerializableBinaryOperator to an ExtendedFunction.Extends a
FunctionDescriptor.SerializableConsumer to an ExtendedFunction.Extends a
FunctionDescriptor.SerializableFunction to an ExtendedFunction.Decorates the default
Function with Serializable, which is required by some distributed frameworks.Decorates the default
Function with Serializable, which is required by some distributed frameworks.Decorates the default
Consumer with Serializable, which is required by some distributed frameworks.Decorates the default
Function with Serializable, which is required by some distributed frameworks.Register for the
ChannelConversions supported for this platform.Register for relevant components of this module.
Executor implementation for the GenericJdbcPlatform.GenericJdbc implementation of the
FilterOperator.Platform implementation for GenericJdbc.GenericJdbc implementation of the
FilterOperator.Implementation of the genetic optimization technique for finding good
LoadProfileEstimators.This app tries to infer good
LoadProfileEstimators for ExecutionOperators using data from an
ExecutionLog.Register for relevant components of this module.
ExecutionOperator that can be run by the GiraphExecutor.Executor for the GiraphPlatform.PageRank
Operator implementation for the GiraphPlatform.Giraph
Platform for Wayang.This
Plugin activates default capabilities of the GiraphPlatform.Mapping from
GlobalMaterializedGroupOperator to FlinkGlobalMaterializedGroupOperator.Mapping from
GlobalMaterializedGroupOperator to JavaGlobalMaterializedGroupOperator.Mapping from
GlobalMaterializedGroupOperator to SparkGlobalMaterializedGroupOperator.This operator groups the elements of a data set into a single data quantum.
This mapping detects combinations of the
GroupByOperator and ReduceOperator and merges them into
a single ReduceByOperator.Mapping from
GlobalReduceOperator to FlinkGlobalReduceOperator.Mapping from
GlobalReduceOperator to JavaGlobalReduceOperator.Mapping from
GlobalReduceOperator to SparkGlobalReduceOperator.This operator groups the elements of a data set and aggregates the groups.
Mapping from
GoogleCloudStorageSource to JavaGoogleClodStorageSource.Mapping from
GroupByOperator to FlinkGroupByOperator.This is the auxiliary GroupBy operator, i.e., it behaves differently depending on its context.
Grouping key used in Query 1.
FileSystem immplementation for the HDFS.TODO: Implement
Mapping from
IEJoinOperator to JavaIEJoinOperator.Mapping from
IEJoinOperator to SparkIEJoinOperator.This operator decides the correct sorting orders for IEJoin
Created by khayyzy on 5/19/16.
This operator applies inequality join on elements of input datasets.
Mapping from
IESelfJoinOperator to JavaIESelfJoinOperator.Mapping from
IESelfJoinOperator to SparkIESelfJoinOperator.This operator applies inequality self join on elements of input datasets.
Context for the optimization of
LoadProfileEstimators.An input slot declares an input of an
Operator.Determines, which
Channels in an ExecutionPlan should be instrumented.Mapping from
IntersectOperator to FlinkIntersectOperator.Mapping from
IntersectOperator to JavaIntersectOperator.Mapping from
IntersectOperator to SparkIntersectOperator.This operator returns the set intersection of elements of input datasets.
Implementation of
LoadEstimator that uses a interval-based cost function.Utilities for the work with
Iterators.Register for relevant components of this module.
This is execution operator that implements the
AmazonS3Source.This is execution operator that implements the
AzureBlobStorageSource.Java implementation of the
CartesianOperator.Defines execution logic to handle a
Channel.Java implementation of the
CoGroupOperator.This is execution operator implements the
TextFileSource.Converts
StreamChannel into a CollectionChannelJava implementation of the
CountOperator.Java implementation of the
DistinctOperator.Java implementation of the
DoWhileOperator.ExecutionContext implementation for the JavaPlatform.Execution operator for the Java platform.
Executor implementation for the JavaPlatform.Java implementation of the
FilterOperator.Java implementation of the
FlatMapOperator.Java implementation of the
GlobalMaterializedGroupOperator.Java implementation of the
GlobalReduceOperator.Java implementation of the
IEJoinOperator.Java implementation of the
IESelfJoinOperator.Java implementation of the
IntersectOperator.Java implementation of the
JoinOperator.Implementation fo the
KafkaTopicSink for the JavaPlatform.Implementation of the
LocalCallbackSink operator for the Java platform.Java implementation of the
LoopOperator.Java implementation of the
MapOperator.Java implementation of the
MapPartitionsOperator.Java implementation of the
MaterializedGroupByOperator.Operator for the JavaPlatform that creates a sequence file.Utility to chunk a
Stream into portions of fixed size.Operator for the JavaPlatform that creates a sequence file.Java implementation of the
PageRankOperator.This is execution operator implements the
ParquetSource.Java implementation of the
JavaRandomSampleOperator.Java implementation of the
ReduceByOperator.Java implementation of the
DoWhileOperator.Java implementation of the
JavaReservoirSampleOperator.Java implementation of the
SortOperator.Implementation fo the
TextFileSink for the JavaPlatform.This is execution operator implements the
TextFileSource.OperatorProfiler for sources.Operator for the JavaPlatform that creates a TSV file.Operator for the JavaPlatform that creates a sequence file.Java implementation of the
UnionAllOperator.Executor implementation for the JdbcPlatformTemplate.Template for JDBC-based
FilterOperator.PostgreSQL implementation for the
JoinOperator.Platform implementation for a JDBC-accessible database.Projects the fields of
Records.Implementation of
Schema that is backed by a JDBC data source.Schema factory that creates a
JdbcSchema.Do not use.
Queryable that gets its data from a table within a JDBC connection.
PostgreSQL implementation for the
TableSource.Describes a job that is to be executed using Wayang.
The Job interface represents a job to be executed in a distributed system.
JobEnd class represents a job's end in a system.
JobStart class represents a job's start in a system.
Mapping from
JoinOperator to FlinkJoinOperator.Mapping from
JoinOperator to JavaJoinOperator.Mapping from
JoinOperator to PostgresJoinOperator.Mapping from
JoinOperator to SparkJoinOperator.This operator returns the cartesian product of elements of input datasets.
This interface prescribes implementing instances to be able to provide itself as a
WayangJsonObj.A
JsonSerializer implementation to serialize JsonSerializables.Utility to deal with
JsonSerializables.Alternative to
JsonSerializable: Externalizes serialization logic.Utilities to deal with JUEL expressions.
Describes the implementation of one
OutputSlot to its occupied InputSlots.This
UnarySink writes all incoming data quanta to a single Kafka topic.Mapping from
KafkaTopicSink to JavaKafkaTopicSink.Mapping from
KafkaTopicSink to SparkKafkaTopicSink.This source reads messages from a Kafka topic and outputs the messages as data units.
Mapping from
KafkaTopicSource to JavaKafkaTopicSource.Mapping from
KafkaTopicSource to SparkKafkaTopicSource.Wrapper for
Wrapper for
KeySelectorUsed by
Configurations to provide some value.Mapping from
KMeansOperator to SparkKMeansOperator.This
PlanEnumerationPruningStrategy follows the idea that we can prune a
PlanImplementation, when there is a further one that is (i) better and (ii) has the exact same
operators with still-to-be-connected Slots.A node wraps a
ChannelInstance and keeps track of predecessor nodes.Callback interface for traversals of
LazyExecutionLineageNodes, thereby accumulating the callback return values.LazyExecutionLineageNode.Aggregator implementation that collects all visited LazyExecutionLineageNode contents.InputStream that is trimmed to a specified size.Mapping from
LinearRegressionOperator to SparkLinearRegressionOperator.Mapping from
LinearSVCOperator to SparkLinearSVCOperator.A tuple of the lineitem table.
Parses a
LineItemTuple from a given CSV line (double quoted, comma-separated).Created by khayyzy on 5/28/16.
An estimate of costs of some executable code expressed as a
ProbabilisticIntervalEstimate.Describes the resource utilization of something executable within an
EstimationContext, such as an
ExecutionOperator or a FunctionDescriptor.Functional interface for lambda expressions to express single-point load estimation functions.
Reflects the (estimated) required resources of an
Operator or FunctionDescriptor.Estimates the
LoadProfile of some executable artifact that takes some input data quanta and produces them.Utilities to deal with
LoadProfileEstimators.Calculates a
TimeEstimate for a link LoadProfile.Converts a
LoadEstimate into a time estimate.This sink executes a callback on each received data unit into a Java
Collection.Mapping from
LocalCallbackSink to FlinkLocalCallbackSink.Mapping from
LocalCallbackSink to JavaLocalCallbackSink.Mapping from
LocalCallbackSink to SparkLocalCallbackSink.FileSystem implementation for the local file system.Evaluates a
Configuration on a ExecutionLog.Provide logging via
Logger.An ML4all logical operator
Logical Operator that uses the [[org.apache.wayang.ml4all.abstraction.plan.ML4allModel]] as input besides the data flowing from its input slot.
Mapping from
LogisticRegressionOperator to SparkLogisticRegressionOperator.Enumerator for
LoopSubplans.Special
OperatorAlternative for LoopHeadOperators.CardinalityPusher implementation for LoopHeadAlternatives.Head of a
LoopSubplan.LoopHeadOperators can be stateful because they might be executed mulitple times.Describes the enumeration of a
LoopSubplan.Goes over a
WayangPlan and isolates its loops.Mapping from
LoopOperator to FlinkLoopOperator.Mapping from
LoopOperator to JavaLoopOperator.Mapping from
LoopOperator to SparkLoopOperator.This operator has three inputs and three outputs.
Wraps a loop of
Operators.CardinalityPusher implementation for LoopSubplans.Key-value cache with "least recently used" eviction strategy.
Starts a profiling run of Spark.
Implementation of
KeyValueProvider that uses a Map to provide a value.Wraps a
Function as a Function.Mapping from
MapOperator to FlinkMapOperator.Mapping from
MapOperator to JavaMapOperator.Mapping from
MapOperator to SparkMapOperator.A map operator represents semantics as they are known from frameworks, such as Spark and Flink.
This descriptor pertains to functions that consume and output multiple data quanta.
Wraps a
Function as a FlatMapFunction.Mapping from
MapPartitionsOperator to FlinkMapPartitionsOperator.Mapping from
MapPartitionsOperator to JavaMapPartitionsOperator.Mapping from
MapPartitionsOperator to SparkMapPartitionsOperator.This operator takes as input potentially multiple input data quanta and outputs multiple input data quanta.
A mapping issues a set of
PlanTransformations that make up the complete mapping.Register for the components provided in the basic plugin.
Register for
Mappings for this platform.Register for the
Mappings supported for this platform.Register for
Mappings for this platform.Mappings for the IEJoinOperator.Register for
Mappings for this platform.Register for the
Mappings supported for this platform.Register for
Mappings for this platform.Register for the
Mappings supported for this platform.Register for
Mappings for this platform.This mapping translates the
GroupByOperator into the MaterializedGroupByOperator.Mapping from
MaterializedGroupByOperator to FlinkMaterializedGroupByOperator.Mapping from
MaterializedGroupByOperator to JavaMaterializedGroupByOperator.Mapping from
MaterializedGroupByOperator to SparkMaterializedGroupByOperator.This operator collocates the data units in a data set w.r.t. a key function.
This class provides an empty implementation of
MathExListener,
which can be extended to create a listener which only needs to handle a subset
of the available methods.This class provides an empty implementation of
MathExVisitor,
which can be extended to create a visitor which only needs to handle a subset
of the available methods.This exception signals a failed
Expression evaluation.This interface defines a complete listener for a parse tree produced by
MathExParser.This interface defines a complete generic visitor for a parse tree produced
by
MathExParser.Measurement captures the value of a metric at a specific time
Custom deserializer for
Measurements
Detects actual subclass of serialized instances and then delegates the deserialization to that subtype.Custom serializer for
Measurements
Detects actual subclass of given instances, encodes this class membership, and then delegates serialization to that subtype.For keeping global variables to access in the logical operators.
Wayang physical plan for ML4all algorithms
A Type that represents a ML model
Deprecated.
Deprecated.
Flattens Tuple2<Record, Record> to Record
Maps keys to multiple values.
Created by khayyzy on 5/28/16.
Created by khayyzy on 5/28/16.
Expression implementation that represents a function that is identified
via its name.LoadProfileEstimator that can host further LoadProfileEstimators.Instruments only outbound
Channels.This
Breakpoint implementation always requests a break unless inside of ExecutionStageLoops.This
UnarySink writes all incoming data quanta to a Object file.Mapping from
ObjectFileSink to FlinkObjectFileSink.Mapping from
ObjectFileSink to JavaObjectFileSink.Mapping from
ObjectFileSink to SparkObjectFileSink.This source reads a text file and outputs the lines as data units.
Mapping from
ObjectFileSource to FlinkObjectFileSource.Mapping from
ObjectFileSource to JavaObjectFileSource.Mapping from
ObjectFileSource to SparkObjectFileSource.Method wrapper that ensures that it is called only once.
An operator is any node that within a
WayangPlan.This operator encapsulates operators that are alternative to each other.
CardinalityPusher implementation for OperatorAlternatives.Helper class for the implementation of the
Operator interface.Utility to de/serialize
Operators.This is not an
Operator in its own right.Utilities to deal with
OperatorContainers.An operator match correlates an
OperatorPattern to an actually matched Operator.An operator pattern matches to a class of operator instances.
Allows to instrument an
JavaExecutionOperator.The result of a single profiling run.
Utilities to create
OperatorProfiler instances.Utilities to create
SparkOperatorProfiler instances.Utility class for
Operators.Manages contextual information required during the optimization of a
WayangPlan.Context for the optimization of
LoadProfileEstimators.Utility methods for the optimization process.
Utility similar to
Optional.Instruments only outbound
Channels.Wrapper for
An output slot declares an output of an
Operator.Basic PageRank implementation.
Master compute associated with
PageRankAlgorithm.Simple VertexInputFormat that supports
PageRankAlgorithmSimple VertexOutputFormat that supports
PageRankAlgorithmSimple VertexReader that supports
PageRankAlgorithmWorker context used with
PageRankAlgorithm.Mapping from
PageRankOperator to FlinkPageRankOperator.Maps
PageRankOperators to GiraphPageRankOperators.Mapping from
PageRankOperator to JavaPageRankOperator.Mapping from
PageRankOperator to org.apache.wayang.spark.operators.graph.SparkPageRankOperator .Creates intial page ranks.
Operator for the PageRank algorithm.Parameters for Basic PageRank implementation.
This source reads a parquet file and outputs the lines as
Record units.Mapping from
ParquetSource to JavaParquetSource.Mapping from
CollectionSource to SparkCollectionSource.This exception signals a failed
Expression evaluation.Captures data of a execution of a set of
ExecutionOperators.JsonSerializer implementation for PartialExecutions.This
Measurement adapts a PartialExecutionMeasurement.Represents a collection of
PlanImplementations that all implement the same section of a WayangPlan (which
is assumed to contain OperatorAlternatives in general).A strategy to prune
PlanImplementations from a PlanEnumeration.The plan partitioner recursively dissects a
WayangPlan into PlanEnumerations and then assembles
them.An
Operator can be activated as soon as all of its inputs are available.Represents a partial execution plan.
This class collects metrics for
WayangPlans.Traverse a plan.
A callback can be invoked during a plan traversal on each traversed node.
A callback can be invoked during a plan traversal on each traversed node.
A platform describes an execution engine that executes
ExecutionOperators.Complete data flow on a single platform, that consists of multiple
ExecutionStages.A plugin contributes the following components to a
WayangContext:
Mappings,
ChannelConversions, and
Configuration properties.
In turn, it may require several Platforms for its operation.Register for relevant components of this module.
PostgreSQL implementation of the
FilterOperator.PostgreSQL implementation of the
JoinOperator.Platform implementation for SQLite3.PostgreSQL implementation of the
FilterOperator.PostgreSQL implementation for the
TableSource.Wraps a
Predicate as a Function.This descriptor pertains to predicates that consume a single data unit.
Mapping from
PredictOperator to SparkPredictOperator.Mapping from
PredictOperator to TensorflowPredictOperator.An value representation that is capable of expressing uncertainty.
An estimate that is capable of expressing uncertainty.
This class provides facilities to save and load
Experiments.Utilities to work with
ProfileDBs.Utility to support finding reasonable
LoadProfileEstimators for JavaExecutionOperators.Utilities to fake Wayang internals etc..
This descriptor pertains to projections.
/**
Mapping from
MapOperator to GenericJdbcProjectionOperator./**
Mapping from
MapOperator to PostgresProjectionOperator.Mapping from
MapOperator to Sqlite3ProjectionOperator.Executor implementation that employs a push model, i.e., data quanta are "pushed"
through the ExecutionStage.Describes the situation where one
JavaRDD is operated on, producing a further JavaRDD.A Type that represents a record with a schema, might be replaced with
something standard like JPA entity.
This is a specific
BasicDataUnitType for Records.This mapping detects combinations of the
GroupByOperator and ReduceOperator and merges them into
a single ReduceByOperator.Mapping from
ReduceByOperator to FlinkReduceByOperator.Mapping from
ReduceByOperator to JavaReduceByOperator.Mapping from
ReduceByOperator to SparkReduceByOperator.This operator groups the elements of a data set and aggregates the groups.
This descriptor pertains to functions that take multiple data units and aggregate them into a single data unit
by means of a tree-like fold, i.e., using a commutative, associative function..
This operator is context dependent: after a
GroupByOperator, it is meant to be a ReduceByOperator;
otherwise, it is a GlobalReduceOperator.This interface provides a reference-counting scheme, e.g., to manage allocated external resources.
Utilities for reflection code.
Mapping from
RepeatOperator to FlinkRepeatOperator.Mapping from
RepeatOperator to JavaRepeatOperator.Mapping from
RepeatOperator to SparkRepeatOperator.This factory takes an
SubplanMatch and derives a replacement Subplan from it.Implementation of the
ReplacementSubplanFactory
that replaces exactly one Operator with exactly one Operator,
where both have the exact same InputSlots and OutputSlot in the exact same order.
Utility to do reservoir sampling with competitions over shared space.
Tuple that is returned by Query 1.
Utility to read from an RRD file.
Mapping from
SampleOperator to FlinkSampleOperator.Mapping from
SampleOperator to JavaRandomSampleOperator.Mapping from
SampleOperator to SparkRandomPartitionSampleOperator.A random sample operator randomly selects its inputs from the input slot and pushes that element to the output slot.
Interface that describes a sampling algorithm that can give bias towards certain elements..
This class checks a
WayangPlan for several sanity criteria:
Subplans must only be used as top-level Operator of OperatorAlternative.Alternative
Subplans must contain more than one Operator
The
SerializableObject interface is a marker interface that indicates
that its implementing classes are serializable.This class executes a stochastic gradient descent optimization on Apache Wayang (incubating).
This class executes a stochastic gradient descent optimization on Apache Wayang (incubating), just like
SGDImpl.This
EstimationContext implementation just stores all required variables without any further logic.This
PlanEnumerationPruningStrategy retains only the best PlanImplementations employing a single
Platform only.OperatorProfiler implementation for sinks.SparkOperatorProfiler implementation for SparkExecutionOperators with one input and no outputs.Abstract class for inputs and outputs to operators.
This mapping can be used to encapsulate subplans by connecting slots (usually against the data flow direction,
i.e., outer output slot to inner output slot, inner input slot to outer input slot).
Mapping from
SortOperator to FlinkSortOperator.Mapping from
SortOperator to JavaSortOperator.Mapping from
SortOperator to SparkSortOperator.This operator sorts the elements in this dataset.
OperatorProfiler implementation for sinks.Register for relevant components of this module.
Spark implementation of the
SparkBernoulliSampleOperator.Takes care of creating a
Broadcast that can be used later on.Converts an uncached
RddChannel into a cached RddChannel.Spark implementation of the
CartesianOperator.Spark implementation of the
JoinOperator.Provides a
Collection to a Spark job.SparkOperatorProfiler for the SparkTextFileSource.Wraps and manages a
JavaSparkContext to avoid steady re-creation.This
Plugin enables to create RDDs.Spark implementation of the
CountOperator.Spark implementation of the
DistinctOperator.Spark implementation of the
DoWhileOperator.ExecutionContext implementation for the SparkPlatform.Execution operator for the
SparkPlatform.Executor implementation for the SparkPlatform.Spark implementation of the
FilterOperator.Spark implementation of the
FlatMapOperator.Spark implementation of the
GlobalMaterializedGroupOperator.Spark implementation of the
GlobalReduceOperator.SparkIEJoinOperator<Type0 extends Comparable<Type0>,Type1 extends Comparable<Type1>,Input extends Copyable<Input>>
Spark implementation of the
IEJoinOperator.SparkIESelfJoinOperator<Type0 extends Comparable<Type0>,Type1 extends Comparable<Type1>,Input extends Copyable<Input>>
Spark implementation of the
IESelfJoinOperator.Spark implementation of the
JoinOperator.Spark implementation of the
JoinOperator.Implementation of the
KafkaTopicSink operator for the Spark platform.Provides a
Collection to a Spark job.A Spark listener implementation that captures events generated during the Spark job execution
and sends them to a Kafka topic for further processing.
Implementation of the
LocalCallbackSink operator for the Spark platform.Spark implementation of the
LoopOperator.Spark implementation of the
MapOperator.Spark implementation of the
MapPartitionsOperator.Spark implementation of the
MaterializedGroupByOperator.Deprecated.
Operator for the SparkPlatform that creates a sequence file.Operator for the SparkPlatform that creates a sequence file.Allows to instrument an
SparkExecutionOperator.The result of a single profiling run.
Provides a
Collection to a Spark job.Platform for Apache Spark.Spark implementation of the
SampleOperator.Spark implementation of the
ReduceByOperator.Spark implementation of the
RepeatOperator.Spark implementation of the
SparkShufflePartitionSampleOperator.Spark implementation of the
SortOperator.SparkOperatorProfiler implementation for SparkExecutionOperators with one input and one output.Implementation of the
TextFileSink operator for the Spark platform.Provides a
Collection to a Spark job.SparkOperatorProfiler for the SparkTextFileSource.Operator for the SparkPlatform that creates a TSV file.Operator for the SparkPlatform that creates a sequence file.SparkOperatorProfiler implementation for SparkExecutionOperators with one input and one output.Spark implementation of the
UnionAllOperator.Spark implementation of the
MapOperator.Register for relevant components of this module.
Implementation of the
FilterOperator for the Sqlite3Platform.Platform implementation for SQLite3.Implementation of the
JdbcProjectionOperator for the Sqlite3Platform.Implementation of the
TableSource for the Sqlite3Platform.Implementation of a
Channel that is given by a SQL query.Describes a specific class of
SqlQueryChannels belonging to a certain JdbcPlatformTemplate.This interface represents a stage in a data processing pipeline.
Builds an
ExecutionPlan from a ExecutionTaskFlow.Criterion to futher split
StageAssignmentTraversal.InterimStage besides precedence.Represents a completed stage in a distributed computing system.
The StageExecutorMetrics class implements the Stage and SerializableObject interfaces and represents
the metrics associated with a stage executed by an executor.
The StageSubmitted class implements the Stage and SerializableObject interfaces to represent a submitted stage in a distributed system.
Utility to create
TimeMeasurements for an Experiment.Controls how conducted experiments will be persisted and loaded
The subject of an
Experiment, e.g., an application or algorithm.A subplan encapsulates connected operators as a single operator.
A subplan match correlates a
SubplanPattern with its actually matched .A subplan pattern describes a class of subplans in a
WayangPlan.UnarySource that provides the tuples from a database table.The Task interface represents a task in a distributed computing or data processing system.
The TaskStatusForRunning enum represents the possible statuses of a task while it is running.
Represents the end status of a task execution.
Represents the getting results of a task execution.
The TaskMetric interface defines the methods used to set and retrieve
performance metrics for a given task.
This class represents the metrics for a task in the Apache Wayang monitoring system.
Represents the start status of a task execution.
Register for relevant components of this module.
TensorflowBatchNorm2D<T extends org.tensorflow.types.family.TNumber>
TensorflowBatchNorm3D<T extends org.tensorflow.types.family.TNumber>
Provides a
Collection to a Tensorflow job.Converts
TensorChannel into a CollectionChannelTensorflowConv2D<T extends org.tensorflow.types.family.TNumber>
TensorflowConv3D<T extends org.tensorflow.types.family.TNumber>
TensorflowConvLSTM2D<T extends org.tensorflow.types.family.TNumber>
TensorflowConvLSTM2D.Cell<T extends org.tensorflow.types.family.TNumber>
Execution operator for the
TensorflowPlatform.Executor implementation for the TensorflowPlatform.TensorflowLinear<T extends org.tensorflow.types.family.TNumber>
Platform for TensorflowThis
UnarySink writes all incoming data quanta to a text file.Mapping from
TextFileSink to FlinkTextFileSink.Mapping from
TextFileSink to JavaTextFileSink.Mapping from
CollectionSource to SparkCollectionSource.This source reads a text file and outputs the lines as data units.
Mapping from
TextFileSource to FlinkTextFileSource.Mapping from
TextFileSource to JavaTextFileSource.Mapping from
CollectionSource to SparkCollectionSource.An estimate of time (in milliseconds) expressed as a
ProbabilisticIntervalEstimate.A
Measurement that captures a certain amount of time in milliseconds.This (linear) converter turns
TimeEstimates into cost estimates.Visitor (as in the Visitor Pattern) for
WayangPlans.Sampling strategy that simulates a tournament between elements.
Main class for the TPC-H app based on Apache Wayang (incubating).
This descriptor pertains to functions that consume a single data unit and output a single data unit.
A helper data structure to manage two values without creating new classes.
A type for tuples.
A type for tuples.
An operation
Expression.Abstract base-class for sinks with a single input.
Abstract base class for sources with a single output.
This operator has a single input and a single output.
Mapping from
UnionAllOperator to FlinkUnionAllOperator.Mapping from
UnionAllOperator to JavaUnionAllOperator.Mapping from
UnionAllOperator to SparkUnionAllOperator.This
Operator creates the union (bag semantics) of two .Used by
Configurations to provide some value.A variable
ExpressionA variable that can be altered by an optimization algorithm.
Class create a that generate aggregatorWrapper
Utility for handling arrays.
Activator for the basic Wayang package.
Activator for graph operations being executed with Wayang's basic operators.
Register for plugins in the module.
Utilities to operate
Collections.This is the entry point for users to work with Wayang.
Class create a that generate aggregatorWrapper
Exception that declares a problem of Wayang.
Wrapper for
FileOutputFormatClass create a for use inside of the LoopOperators
JSONArray is the wrapper for the
ArrayNode to enable the
easy access to Json data
TODO: the java doc is not done because is missing implementation and it performed some
modification on the codeJSONObject is the wrapper for the
ObjectNode to enable the
easy access to the json data
TODO: the java doc is not done because is missing implementation and it performed some
modification on the codeIs a Wrapper for used in the criterion of the Loops
A Wayang plan consists of a set of
Operators.Rule that converts
LogicalSort to Wayang convention
WayangSortImplementation of
Value of flink for use in WayangExample Apache Wayang (incubating) App that does a word count -- the Hello World of Map/Reduce-like systems.
TODO: Implement
Mapping from
ZipWithIdOperator to FlinkZipWithIdOperator.Mapping from
ZipWithIdMapping to a subplan.Mapping from
ZipWithIdOperator to SparkZipWithIdOperator.This operators attaches a unique ID to each input data quantum.