Package org.apache.wayang.profiler.spark

Class SparkCollectionSourceProfiler

java.lang.Object

org.apache.wayang.profiler.spark.SparkOperatorProfiler

org.apache.wayang.profiler.spark.SparkSourceProfiler

org.apache.wayang.profiler.spark.SparkCollectionSourceProfiler

public class SparkCollectionSourceProfiler
extends SparkSourceProfiler

SparkOperatorProfiler for the SparkTextFileSource.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

SparkOperatorProfiler.Result

Field Summary

Fields inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

cpuMhz, dataQuantumGenerators, executionPaddingTime, functionCompiler, inputCardinalities, logger, numCoresPerMachine, numMachines, numPartitions, operator, operatorGenerator, sparkExecutor

Constructor Summary

Constructors

Constructor	Description
SparkCollectionSourceProfiler(Configuration configuration, java.util.function.Supplier<T> dataQuantumGenerator, DataSetType<T> outputType)

Method Summary

Modifier and Type	Method	Description
void	cleanUp()	Override this method to implement any clean-up logic.
protected void	prepareInput(int inputIndex, long inputCardinality)

Methods inherited from class org.apache.wayang.profiler.spark.SparkSourceProfiler

executeOperator

Methods inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

createChannelInstance, createChannelInstance, evaluate, partition, prepare, prepareInputRdd, prepareInputRddInDriver, prepareInputRddInWorker, provideCpuCycles, provideDiskBytes, provideNetworkBytes, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkCollectionSourceProfiler

public SparkCollectionSourceProfiler(Configuration configuration,
                                     java.util.function.Supplier<T> dataQuantumGenerator,
                                     DataSetType<T> outputType)

Method Detail

prepareInput

protected void prepareInput(int inputIndex,
                            long inputCardinality)

Specified by:

prepareInput in class SparkOperatorProfiler

cleanUp

public void cleanUp()

Description copied from class: SparkOperatorProfiler

Override this method to implement any clean-up logic.

Overrides:

cleanUp in class SparkOperatorProfiler