Package org.apache.wayang.profiler.spark

Class SparkSourceProfiler

java.lang.Object

org.apache.wayang.profiler.spark.SparkOperatorProfiler

org.apache.wayang.profiler.spark.SparkSourceProfiler

Direct Known Subclasses:

SparkCollectionSourceProfiler, SparkTextFileSourceProfiler

public abstract class SparkSourceProfiler
extends SparkOperatorProfiler

SparkOperatorProfiler implementation for SparkExecutionOperators with one input and one output.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

SparkOperatorProfiler.Result

Field Summary

Fields inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

cpuMhz, dataQuantumGenerators, executionPaddingTime, functionCompiler, inputCardinalities, logger, numCoresPerMachine, numMachines, numPartitions, operator, operatorGenerator, sparkExecutor

Constructor Summary

Constructors

Constructor	Description
SparkSourceProfiler(java.util.function.Supplier<SparkExecutionOperator> operatorGenerator, Configuration configuration, java.util.function.Supplier<?> dataQuantumGenerator)

Method Summary

Modifier and Type	Method	Description
protected SparkOperatorProfiler.Result	executeOperator()	Executes the profiling task.

Methods inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

cleanUp, createChannelInstance, createChannelInstance, evaluate, partition, prepare, prepareInput, prepareInputRdd, prepareInputRddInDriver, prepareInputRddInWorker, provideCpuCycles, provideDiskBytes, provideNetworkBytes, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkSourceProfiler

public SparkSourceProfiler(java.util.function.Supplier<SparkExecutionOperator> operatorGenerator,
                           Configuration configuration,
                           java.util.function.Supplier<?> dataQuantumGenerator)

Method Detail

executeOperator

protected SparkOperatorProfiler.Result executeOperator()

Description copied from class: SparkOperatorProfiler

Executes the profiling task. Requires that this instance is prepared.

Specified by:

executeOperator in class SparkOperatorProfiler