org.apache.spark.streaming.dstream

Class DStreamCheckpointData<T>

Object

org.apache.spark.streaming.dstream.DStreamCheckpointData<T>

All Implemented Interfaces:

java.io.Serializable, Logging

Direct Known Subclasses:

DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData, FileInputDStream.FileInputDStreamCheckpointData

public class DStreamCheckpointData<T>
extends Object
implements scala.Serializable, Logging

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DStreamCheckpointData(DStream<T> dstream, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
void	cleanup(Time time) Cleanup old checkpoint data.
void	restore() Restore the checkpoint data.
String	toString()
void	update(Time time) Updates the checkpoint data of the DStream.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

DStreamCheckpointData

public DStreamCheckpointData(DStream<T> dstream,
                     scala.reflect.ClassTag<T> evidence$1)

Method Detail

update

public void update(Time time)

Updates the checkpoint data of the DStream. This gets called every time the graph checkpoint is initiated. Default implementation records the checkpoint files to which the generate RDDs of the DStream has been saved.

cleanup

public void cleanup(Time time)

Cleanup old checkpoint data. This gets called after a checkpoint of time has been written to the checkpoint directory.

restore

public void restore()

Restore the checkpoint data. This gets called once when the DStream graph (along with its DStreams) are being restored from a graph checkpoint file. Default implementation restores the RDDs from their checkpoint files.

toString

public String toString()

Overrides:

toString in class Object