Package org.apache.wayang.spark.platform

Class SparkPlatform

java.lang.Object

org.apache.wayang.core.platform.Platform

org.apache.wayang.spark.platform.SparkPlatform

public class SparkPlatform
extends Platform

Platform for Apache Spark.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	INITIALIZATION_MS_CONFIG_KEY

Fields inherited from class org.apache.wayang.core.platform.Platform

jsonSerializer

Method Summary

Modifier and Type	Method	Description
void	configureDefaults(Configuration configuration)	Configure default settings for this instance, e.g., to be able to create LoadProfileToTimeConverters.
LoadProfileToTimeConverter	createLoadProfileToTimeConverter(Configuration configuration)
TimeToCostConverter	createTimeToCostConverter(Configuration configuration)	Creates a TimeToCostConverter for this instance.
Executor.Factory	getExecutorFactory()
long	getInitializeMillis(Configuration configuration)	Get the time necessary to initialize this instance and use it for execution.
static SparkPlatform	getInstance()
SparkContextReference	getSparkContext(Job job)	Configures the single maintained JavaSparkContext according to the job and returns it.
void	warmUp(Configuration configuration)	Warm up this instance.

Methods inherited from class org.apache.wayang.core.platform.Platform

createExecutor, getConfigurationName, getName, isSinglePlatformExecutionPossible, load, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

INITIALIZATION_MS_CONFIG_KEY

public static final String INITIALIZATION_MS_CONFIG_KEY

See Also:

• Constant Field Values

Method Details

getInstance

public static SparkPlatform getInstance()

getSparkContext

public SparkContextReference getSparkContext(Job job)

Configures the single maintained JavaSparkContext according to the job and returns it.

Returns:

a SparkContextReference wrapping the JavaSparkContext

configureDefaults

public void configureDefaults(Configuration configuration)

Description copied from class: Platform

Configure default settings for this instance, e.g., to be able to create LoadProfileToTimeConverters.

Specified by:

configureDefaults in class Platform

Parameters:

configuration - that should be configured

createLoadProfileToTimeConverter

public LoadProfileToTimeConverter createLoadProfileToTimeConverter(Configuration configuration)

Specified by:

createLoadProfileToTimeConverter in class Platform

Returns:

a default LoadProfileToTimeConverter

createTimeToCostConverter

public TimeToCostConverter createTimeToCostConverter(Configuration configuration)

Description copied from class: Platform

Creates a TimeToCostConverter for this instance.

Specified by:

createTimeToCostConverter in class Platform

Parameters:

configuration - configures the TimeToCostConverter

Returns:

the TimeToCostConverter

getExecutorFactory

public Executor.Factory getExecutorFactory()

Specified by:

getExecutorFactory in class Platform

warmUp

public void warmUp(Configuration configuration)

Description copied from class: Platform

Warm up this instance.

Overrides:

warmUp in class Platform

Parameters:

configuration - used to warm up

getInitializeMillis

public long getInitializeMillis(Configuration configuration)

Description copied from class: Platform

Get the time necessary to initialize this instance and use it for execution.

Overrides:

getInitializeMillis in class Platform

Returns:

the milliseconds required to initialize this instance