Package org.apache.wayang.profiler.spark

Class SparkTextFileSourceProfiler

java.lang.Object

org.apache.wayang.profiler.spark.SparkOperatorProfiler

org.apache.wayang.profiler.spark.SparkSourceProfiler

org.apache.wayang.profiler.spark.SparkTextFileSourceProfiler

public class SparkTextFileSourceProfiler
extends SparkSourceProfiler

SparkOperatorProfiler for the SparkTextFileSource.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

SparkOperatorProfiler.Result

Field Summary

Fields inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

cpuMhz, dataQuantumGenerators, executionPaddingTime, functionCompiler, inputCardinalities, logger, numCoresPerMachine, numMachines, numPartitions, operator, operatorGenerator, sparkExecutor

Constructor Summary

Constructors

Constructor	Description
SparkTextFileSourceProfiler(Configuration configuration, java.util.function.Supplier<?> dataQuantumGenerator)

Method Summary

Modifier and Type	Method	Description
void	cleanUp()	Override this method to implement any clean-up logic.
protected void	prepareInput(int inputIndex, long inputCardinality)

Methods inherited from class org.apache.wayang.profiler.spark.SparkSourceProfiler

executeOperator

Methods inherited from class org.apache.wayang.profiler.spark.SparkOperatorProfiler

createChannelInstance, createChannelInstance, evaluate, partition, prepare, prepareInputRdd, prepareInputRddInDriver, prepareInputRddInWorker, provideCpuCycles, provideDiskBytes, provideNetworkBytes, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkTextFileSourceProfiler

public SparkTextFileSourceProfiler(Configuration configuration,
                                   java.util.function.Supplier<?> dataQuantumGenerator)

Method Detail

prepareInput

protected void prepareInput(int inputIndex,
                            long inputCardinality)

Specified by:

prepareInput in class SparkOperatorProfiler

cleanUp

public void cleanUp()

Description copied from class: SparkOperatorProfiler

Override this method to implement any clean-up logic.

Overrides:

cleanUp in class SparkOperatorProfiler