Package org.apache.wayang.basic.operators

Class AzureBlobStorageSource

java.lang.Object

org.apache.wayang.core.plan.wayangplan.OperatorBase

org.apache.wayang.core.plan.wayangplan.UnarySource<String>

org.apache.wayang.basic.operators.AzureBlobStorageSource

All Implemented Interfaces:

Serializable, ActualOperator, ElementaryOperator, Operator

Direct Known Subclasses:

JavaAzureBlobStorageSource

public class AzureBlobStorageSource
extends UnarySource<String>

This source reads a blob file stored in Azure Blob Storage and outputs the lines as data units.

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	AzureBlobStorageSource.CardinalityEstimator	Custom CardinalityEstimator for FlatMapOperators.

Nested classes/interfaces inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

OperatorBase.GsonSerializer

Field Summary

Fields inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

inputSlots, outputSlots, STANDARD_OPERATOR_ARGS

Fields inherited from interface org.apache.wayang.core.plan.wayangplan.Operator

FIRST_EPOCH

Constructor Summary

Constructors

Constructor	Description
AzureBlobStorageSource(String storageContainer, String blobName, String filePathToCredentialsFile)
AzureBlobStorageSource(String storageContainer, String blobName, String filePathToCredentialsFile, String encoding)
AzureBlobStorageSource(AzureBlobStorageSource that)	Copies an instance (exclusive of broadcasts).

Method Summary

Modifier and Type	Method	Description
OptionalLong	getBlobByteSize()
String	getBlobName()
String	getEncoding()
String	getFilePathToCredentialsFile()
InputStream	getInputStream()	Retrieves an InputStream to the specified Azure blob file.
String	getStorageContainer()

Methods inherited from class org.apache.wayang.core.plan.wayangplan.UnarySource

getOutput, getType

Methods inherited from class org.apache.wayang.core.plan.wayangplan.OperatorBase

accept, addBroadcastInput, addTargetPlatform, at, collectMappedInputSlots, collectMappedOutputSlots, copy, createCopy, getAllInputs, getAllOutputs, getCardinalityEstimator, getContainer, getEpoch, getName, getOriginal, getSimpleClassName, getTargetPlatforms, isAuxiliary, isSupportingBroadcastInputs, propagateInputCardinality, propagateOutputCardinality, setAuxiliary, setCardinalityEstimator, setContainer, setEpoch, setName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.ActualOperator

accept

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.ElementaryOperator

createCardinalityEstimator, getCardinalityEstimator, isAuxiliary, setAuxiliary, setCardinalityEstimator

Methods inherited from interface org.apache.wayang.core.plan.wayangplan.Operator

addBroadcastInput, addTargetPlatform, broadcastTo, broadcastTo, collectMappedInputSlots, collectMappedOutputSlots, connectTo, connectTo, getAllInputs, getAllOutputs, getCardinalityPusher, getContainer, getEffectiveOccupant, getEffectiveOccupant, getEpoch, getEstimationContextProperties, getForwards, getInnermostLoop, getInput, getInput, getLoopStack, getName, getNumBroadcastInputs, getNumInputs, getNumOutputs, getNumRegularInputs, getOuterInputSlot, getOutermostInputSlot, getOutermostOutputSlots, getOutput, getOutput, getParent, getTargetPlatforms, isAlternative, isConversion, isElementary, isExecutionOperator, isFeedbackInput, isFeedforwardOutput, isLoopHead, isLoopSubplan, isOwnerOf, isReading, isSink, isSource, isSubplan, isSupportingBroadcastInputs, isUnconnected, propagateInputCardinality, propagateOutputCardinality, propagateOutputCardinality, replaceWith, setContainer, setEpoch, setInput, setName, setOutput

Constructor Details

AzureBlobStorageSource

public AzureBlobStorageSource(String storageContainer,
 String blobName,
 String filePathToCredentialsFile)

AzureBlobStorageSource

public AzureBlobStorageSource(String storageContainer,
 String blobName,
 String filePathToCredentialsFile,
 String encoding)

AzureBlobStorageSource

public AzureBlobStorageSource(AzureBlobStorageSource that)

Copies an instance (exclusive of broadcasts).

Parameters:

that - that should be copied

Method Details

getEncoding

public String getEncoding()

getStorageContainer

public String getStorageContainer()

getBlobName

public String getBlobName()

getFilePathToCredentialsFile

public String getFilePathToCredentialsFile()

getBlobByteSize

public OptionalLong getBlobByteSize()

Returns:

the total size of the bytes in the Blob file. Returns empty OptionalLong if an exception is caught.

getInputStream

public InputStream getInputStream()
                           throws Exception

Retrieves an InputStream to the specified Azure blob file.

Returns:

InputStream to the Blob file.

Throws:

Exception - if an error occurs during Azure client creation or file rertrieval.