org.apache.spark.storage

Class DiskBlockObjectWriter

Object

org.apache.spark.storage.BlockObjectWriter

org.apache.spark.storage.DiskBlockObjectWriter

All Implemented Interfaces:

Logging

public class DiskBlockObjectWriter
extends BlockObjectWriter
implements Logging

BlockObjectWriter which writes directly to a file on disk. Appends to the given file.

Constructor Summary

Constructors

Constructor and Description
DiskBlockObjectWriter(BlockId blockId, java.io.File file, Serializer serializer, int bufferSize, scala.Function1<java.io.OutputStream,java.io.OutputStream> compressStream, boolean syncWrites, org.apache.spark.executor.ShuffleWriteMetrics writeMetrics)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	commitAndClose() Flush the partial writes and commit them as a single atomic block.
FileSegment	fileSegment() Returns the file segment of committed data that this Writer has written.
void	flush()
boolean	isOpen()
BlockObjectWriter	open()
void	revertPartialWritesAndClose() Reverts writes that haven't been flushed yet.
void	write(Object value) Writes an object.

Methods inherited from class org.apache.spark.storage.BlockObjectWriter

blockId

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

DiskBlockObjectWriter

public DiskBlockObjectWriter(BlockId blockId,
                     java.io.File file,
                     Serializer serializer,
                     int bufferSize,
                     scala.Function1<java.io.OutputStream,java.io.OutputStream> compressStream,
                     boolean syncWrites,
                     org.apache.spark.executor.ShuffleWriteMetrics writeMetrics)

Method Detail

open

public BlockObjectWriter open()

Specified by:

open in class BlockObjectWriter

close

public void close()

Specified by:

close in class BlockObjectWriter

isOpen

public boolean isOpen()

Specified by:

isOpen in class BlockObjectWriter

commitAndClose

public void commitAndClose()

Description copied from class: BlockObjectWriter

Flush the partial writes and commit them as a single atomic block.

Specified by:

commitAndClose in class BlockObjectWriter

revertPartialWritesAndClose

public void revertPartialWritesAndClose()

Description copied from class: BlockObjectWriter

Reverts writes that haven't been flushed yet. Callers should invoke this function when there are runtime exceptions. This method will not throw, though it may be unsuccessful in truncating written data.

Specified by:

revertPartialWritesAndClose in class BlockObjectWriter

write

public void write(Object value)

Description copied from class: BlockObjectWriter

Writes an object.

Specified by:

write in class BlockObjectWriter

fileSegment

public FileSegment fileSegment()

Description copied from class: BlockObjectWriter

Returns the file segment of committed data that this Writer has written. This is only valid after commitAndClose() has been called.

Specified by:

fileSegment in class BlockObjectWriter

flush

public void flush()