org.apache.flink.runtime.io.network.buffer

Class NetworkBufferPool

java.lang.Object

org.apache.flink.runtime.io.network.buffer.NetworkBufferPool

All Implemented Interfaces:

org.apache.flink.core.memory.MemorySegmentProvider, AvailabilityProvider, BufferPoolFactory

public class NetworkBufferPool
extends Object
implements BufferPoolFactory, org.apache.flink.core.memory.MemorySegmentProvider, AvailabilityProvider

The NetworkBufferPool is a fixed size pool of MemorySegment instances for the network stack.

The NetworkBufferPool creates LocalBufferPools from which the individual tasks draw the buffers for the network data transfer. When new local buffer pools are created, the NetworkBufferPool dynamically redistributes the buffers between the pools.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AvailabilityProvider.AvailabilityHelper

Field Summary

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Constructor Summary

Constructors

Constructor and Description
NetworkBufferPool(int numberOfSegmentsToAllocate, int segmentSize, int numberOfSegmentsToRequest)
NetworkBufferPool(int numberOfSegmentsToAllocate, int segmentSize, int numberOfSegmentsToRequest, java.time.Duration requestSegmentsTimeout) Allocates all MemorySegment instances managed by this pool.

Method Summary

Modifier and Type	Method and Description
int	countBuffers()
BufferPool	createBufferPool(int numRequiredBuffers, int maxUsedBuffers) Tries to create a buffer pool, which is guaranteed to provide at least the number of required buffers.
BufferPool	createBufferPool(int numRequiredBuffers, int maxUsedBuffers, BufferPoolOwner bufferPoolOwner) Tries to create a buffer pool with an owner, which is guaranteed to provide at least the number of required buffers.
void	destroy()
void	destroyAllBufferPools() Destroys all buffer pools that allocate their buffers from this buffer pool (created via createBufferPool(int, int)).
void	destroyBufferPool(BufferPool bufferPool) Destroy callback for updating factory book keeping.
CompletableFuture<?>	getAvailableFuture() Returns a future that is completed when there are free segments in this pool.
int	getNumberOfAvailableMemorySegments()
int	getNumberOfRegisteredBufferPools()
int	getTotalNumberOfMemorySegments()
boolean	isDestroyed()
void	recycle(org.apache.flink.core.memory.MemorySegment segment)
void	recycleMemorySegments(Collection<org.apache.flink.core.memory.MemorySegment> segments)
org.apache.flink.core.memory.MemorySegment	requestMemorySegment()
List<org.apache.flink.core.memory.MemorySegment>	requestMemorySegments()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

isApproximatelyAvailable, isAvailable

Constructor Detail

NetworkBufferPool

@VisibleForTesting
public NetworkBufferPool(int numberOfSegmentsToAllocate,
                                            int segmentSize,
                                            int numberOfSegmentsToRequest)

NetworkBufferPool

public NetworkBufferPool(int numberOfSegmentsToAllocate,
                         int segmentSize,
                         int numberOfSegmentsToRequest,
                         java.time.Duration requestSegmentsTimeout)

Allocates all MemorySegment instances managed by this pool.

Method Detail

requestMemorySegment

@Nullable
public org.apache.flink.core.memory.MemorySegment requestMemorySegment()

recycle

public void recycle(org.apache.flink.core.memory.MemorySegment segment)

requestMemorySegments

public List<org.apache.flink.core.memory.MemorySegment> requestMemorySegments()
                                                                       throws IOException

Specified by:

requestMemorySegments in interface org.apache.flink.core.memory.MemorySegmentProvider

Throws:

IOException

recycleMemorySegments

public void recycleMemorySegments(Collection<org.apache.flink.core.memory.MemorySegment> segments)
                           throws IOException

Specified by:

recycleMemorySegments in interface org.apache.flink.core.memory.MemorySegmentProvider

Throws:

IOException

destroy

public void destroy()

isDestroyed

public boolean isDestroyed()

getTotalNumberOfMemorySegments

public int getTotalNumberOfMemorySegments()

getNumberOfAvailableMemorySegments

public int getNumberOfAvailableMemorySegments()

getNumberOfRegisteredBufferPools

public int getNumberOfRegisteredBufferPools()

countBuffers

public int countBuffers()

getAvailableFuture

public CompletableFuture<?> getAvailableFuture()

Returns a future that is completed when there are free segments in this pool.

Specified by:

getAvailableFuture in interface AvailabilityProvider

Returns:

a future that is completed if the respective provider is available.

createBufferPool

public BufferPool createBufferPool(int numRequiredBuffers,
                                   int maxUsedBuffers)
                            throws IOException

Description copied from interface: BufferPoolFactory

Tries to create a buffer pool, which is guaranteed to provide at least the number of required buffers.

The buffer pool is of dynamic size with at least numRequiredBuffers buffers.

Specified by:

createBufferPool in interface BufferPoolFactory

Parameters:

numRequiredBuffers - minimum number of network buffers in this pool

maxUsedBuffers - maximum number of network buffers this pool offers

Throws:

IOException

createBufferPool

public BufferPool createBufferPool(int numRequiredBuffers,
                                   int maxUsedBuffers,
                                   BufferPoolOwner bufferPoolOwner)
                            throws IOException

Description copied from interface: BufferPoolFactory

Tries to create a buffer pool with an owner, which is guaranteed to provide at least the number of required buffers.

The buffer pool is of dynamic size with at least numRequiredBuffers buffers.

Specified by:

createBufferPool in interface BufferPoolFactory

Parameters:

numRequiredBuffers - minimum number of network buffers in this pool

maxUsedBuffers - maximum number of network buffers this pool offers

bufferPoolOwner - the owner of this buffer pool to release memory when needed

Throws:

IOException

destroyBufferPool

public void destroyBufferPool(BufferPool bufferPool)
                       throws IOException

Description copied from interface: BufferPoolFactory

Destroy callback for updating factory book keeping.

Specified by:

destroyBufferPool in interface BufferPoolFactory

Throws:

IOException

destroyAllBufferPools

public void destroyAllBufferPools()

Destroys all buffer pools that allocate their buffers from this buffer pool (created via createBufferPool(int, int)).