Detailed Description

Definition at line 79 of file dispatcher.hpp.

Inheritance diagram for Dispatcher:

Collaboration diagram for Dispatcher:

#include <dispatcher.hpp>

Classes
class	MpiAsync
	This is the big answer to what happens when an MPI async request is signaled as complete: it unifies all possible async requests, including the reference counts they hold on the appropriate buffers, and dispatches the correct callbacks when done. More...

struct	Watch
	callback vectors per peer More...

Public Types
using	Callback = AsyncCallback
	type for file descriptor readiness callbacks More...

Public Member Functions
	Dispatcher (size_t group_size)
	constructor More...

	~Dispatcher ()
	destructor More...

void	AddAsyncRequest (const MPI_Request &req, const AsyncRequestCallback &callback)

void	AddRead (net::Connection &c, const Callback &read_cb) final
	Register a buffered read callback and a default exception callback. More...

void	AddWrite (net::Connection &, const Callback &) final
	Register a buffered write callback and a default exception callback. More...

void	AsyncRead (net::Connection &c, uint32_t seq, size_t size, const AsyncReadBufferCallback &done_cb=AsyncReadBufferCallback()) final
	asynchronously read n bytes and deliver them to the callback More...

void	AsyncRead (net::Connection &c, uint32_t seq, size_t size, data::PinnedByteBlockPtr &&block, const AsyncReadByteBlockCallback &done_cb) final
	asynchronously read the full ByteBlock and deliver it to the callback More...

void	AsyncWrite (net::Connection &c, uint32_t seq, Buffer &&buffer, const AsyncWriteCallback &done_cb=AsyncWriteCallback()) final

void	AsyncWrite (net::Connection &c, uint32_t seq, data::PinnedBlock &&block, const AsyncWriteCallback &done_cb=AsyncWriteCallback()) final

void	Cancel (net::Connection &c) final
	Cancel all callbacks on a given peer. More...

void	DispatchOne (const std::chrono::milliseconds &timeout) final
	Run one iteration of dispatching using MPI_Iprobe(). More...

void	Interrupt () final
	Interrupt does nothing. More...

MPI_Request	IRecv (Connection &c, uint32_t seq, void *data, size_t size)

MPI_Request	ISend (Connection &c, uint32_t seq, const void *data, size_t size)

void	PerformAsync (MpiAsync &&a)
	Issue the encapsulated request to the MPI layer. More...

void	PumpRecvQueue (int peer)
	Check recv queue and perform waiting requests. More...

void	PumpSendQueue (int peer)
	Check send queue and perform waiting requests. More...

void	QueueAsyncRecv (net::Connection &c, MpiAsync &&a)
	Enqueue and run the encapsulated result. More...

void	QueueAsyncSend (net::Connection &c, MpiAsync &&a)
	Enqueue and run the encapsulated result. More...

void	SetExcept (net::Connection &c, const Callback &except_cb)
	Register a buffered write callback and a default exception callback. More...

Public Member Functions inherited from Dispatcher
	Dispatcher ()=default
	default constructor More...

	Dispatcher (const Dispatcher &)=delete
	non-copyable: delete copy-constructor More...

Dispatcher &	operator= (const Dispatcher &)=delete
	non-copyable: delete assignment operator More...

void	AddTimer (const std::chrono::milliseconds &timeout, const TimerCallback &cb)
	Register a relative timeout callback. More...

void	AsyncWriteCopy (Connection &c, uint32_t seq, const void *buffer, size_t size, const AsyncWriteCallback &done_cb=AsyncWriteCallback())

void	AsyncWriteCopy (Connection &c, uint32_t seq, const std::string &str, const AsyncWriteCallback &done_cb=AsyncWriteCallback())

void	Dispatch ()
	Dispatch one or more events. More...

void	Loop ()
	Loop over Dispatch() until terminate_ flag is set. More...

void	Terminate ()

bool	HasAsyncWrites () const
	Check whether there are still AsyncWrite()s in the queue. More...

Static Private Member Functions
static bool	DefaultExceptionCallback ()
	Default exception handler. More...

Private Attributes
std::vector< MpiAsync >	mpi_async_

std::vector< int >	mpi_async_out_
	array of output integer of finished requests for MPI_Testsome(). More...

std::vector< MPI_Request >	mpi_async_requests_
	array of current async MPI_Request for MPI_Testsome(). More...

std::vector< MPI_Status >	mpi_status_out_
	array of output MPI_Status of finished requests for MPI_Testsome(). More...

std::vector< size_t >	recv_active_
	number of active requests More...

std::deque< std::deque< MpiAsync > >	recv_queue_
	queue of delayed requests for each peer More...

std::vector< size_t >	send_active_
	number of active requests More...

std::deque< std::deque< MpiAsync > >	send_queue_
	queue of delayed requests for each peer More...

std::vector< Watch >	watch_
	callback watch vector More...

size_t	watch_active_ { 0 }
	counter of active watches More...

Static Private Attributes
static constexpr bool	debug = false

Additional Inherited Members
Protected Types inherited from Dispatcher
using	TimerPQ = std::priority_queue< Timer, std::vector< Timer, mem::GPoolAllocator< Timer > > >
	priority queue of timer callbacks More...

Static Protected Member Functions inherited from Dispatcher
static bool	ExceptionCallback (Connection &c)
	Default exception handler. More...

Protected Attributes inherited from Dispatcher
std::deque< AsyncReadBuffer, mem::GPoolAllocator< AsyncReadBuffer > >	async_read_
	deque of asynchronous readers More...

std::deque< AsyncReadByteBlock, mem::GPoolAllocator< AsyncReadByteBlock > >	async_read_block_
	deque of asynchronous readers More...

std::deque< AsyncWriteBuffer, mem::GPoolAllocator< AsyncWriteBuffer > >	async_write_
	deque of asynchronous writers More...

std::deque< AsyncWriteBlock, mem::GPoolAllocator< AsyncWriteBlock > >	async_write_block_
	deque of asynchronous writers More...

std::atomic< bool >	terminate_ { false }
	true if dispatcher needs to stop More...

TimerPQ	timer_pq_

Member Typedef Documentation

◆ Callback

using Callback = AsyncCallback

type for file descriptor readiness callbacks

Definition at line 87 of file dispatcher.hpp.

Constructor & Destructor Documentation

◆ Dispatcher()

Dispatcher ( size_t group_size )

constructor

Definition at line 32 of file dispatcher.cpp.

References Dispatcher::recv_active_, Dispatcher::recv_queue_, Dispatcher::send_active_, Dispatcher::send_queue_, and Dispatcher::watch_.

◆ ~Dispatcher()

~Dispatcher ( )

virtual

destructor

Reimplemented from Dispatcher.

Definition at line 44 of file dispatcher.cpp.

References LOG, LOG1, Dispatcher::mpi_async_, and Dispatcher::mpi_async_requests_.

Member Function Documentation

◆ AddAsyncRequest()

void AddAsyncRequest	(	const MPI_Request &	req,
		const AsyncRequestCallback &	callback
	)

Definition at line 105 of file dispatcher.cpp.

References LOG, Dispatcher::mpi_async_, Dispatcher::mpi_async_out_, Dispatcher::mpi_async_requests_, and Dispatcher::mpi_status_out_.

Referenced by Group::Barrier(), and Group::WaitForRequest().

◆ AddRead()

void AddRead	(	net::Connection &	c,
		const Callback &	read_cb
	)

inlinefinalvirtual

Register a buffered read callback and a default exception callback.

Implements Dispatcher.

Definition at line 96 of file dispatcher.hpp.

References Connection::peer().

◆ AddWrite()

void AddWrite	(	net::Connection &	,
		const Callback &
	)

inlinefinalvirtual

Register a buffered write callback and a default exception callback.

Implements Dispatcher.

Definition at line 107 of file dispatcher.hpp.

◆ AsyncRead() [1/2]

void AsyncRead	(	net::Connection &	c,
		uint32_t	,
		size_t	size,
		const AsyncReadBufferCallback &	done_cb = `AsyncReadBufferCallback()`
	)

inlinefinalvirtual

asynchronously read n bytes and deliver them to the callback

Reimplemented from Dispatcher.

Definition at line 175 of file dispatcher.hpp.

References Connection::IsValid().

◆ AsyncRead() [2/2]

void AsyncRead	(	net::Connection &	c,
		uint32_t	,
		size_t	size,
		data::PinnedByteBlockPtr &&	block,
		const AsyncReadByteBlockCallback &	done_cb
	)

inlinefinalvirtual

asynchronously read the full ByteBlock and deliver it to the callback

Reimplemented from Dispatcher.

Definition at line 188 of file dispatcher.hpp.

References Connection::IsValid().

◆ AsyncWrite() [1/2]

void AsyncWrite	(	net::Connection &	c,
		uint32_t	,
		Buffer &&	buffer,
		const AsyncWriteCallback &	done_cb = `AsyncWriteCallback()`
	)

inlinefinalvirtual

asynchronously write buffer and callback when delivered. The buffer is MOVED into the async writer.

Reimplemented from Dispatcher.

Definition at line 149 of file dispatcher.hpp.

References Connection::IsValid().

◆ AsyncWrite() [2/2]

void AsyncWrite	(	net::Connection &	c,
		uint32_t	,
		data::PinnedBlock &&	block,
		const AsyncWriteCallback &	done_cb = `AsyncWriteCallback()`
	)

inlinefinalvirtual

asynchronously write buffer and callback when delivered. The buffer is MOVED into the async writer.

Reimplemented from Dispatcher.

Definition at line 162 of file dispatcher.hpp.

References Connection::IsValid().

◆ Cancel()

void Cancel ( net::Connection & c )

inlinefinalvirtual

Cancel all callbacks on a given peer.

Implements Dispatcher.

Definition at line 124 of file dispatcher.hpp.

References Dispatcher::Watch::active, Dispatcher::Watch::except_cb, LOG, Connection::peer(), and Dispatcher::Watch::read_cb.

◆ DefaultExceptionCallback()

static bool DefaultExceptionCallback ( )

inlinestaticprivate

Default exception handler.

Definition at line 242 of file dispatcher.hpp.

◆ DispatchOne()

void DispatchOne ( const std::chrono::milliseconds & timeout )

finalvirtual

Run one iteration of dispatching using MPI_Iprobe().

Implements Dispatcher.

Definition at line 265 of file dispatcher.cpp.

References Dispatcher::MpiAsync::connection(), die_unless, LOG, Dispatcher::mpi_async_, Dispatcher::mpi_async_out_, Dispatcher::mpi_async_requests_, Dispatcher::mpi_status_out_, Connection::peer(), Dispatcher::PumpRecvQueue(), Dispatcher::PumpSendQueue(), Dispatcher::MpiAsync::READ_BUFFER, Dispatcher::MpiAsync::READ_BYTE_BLOCK, Dispatcher::recv_active_, Dispatcher::send_active_, sLOG, sLOG0, Dispatcher::MpiAsync::type_, Dispatcher::watch_, Dispatcher::watch_active_, Dispatcher::MpiAsync::WRITE_BLOCK, and Dispatcher::MpiAsync::WRITE_BUFFER.

◆ Interrupt()

void Interrupt ( )

inlinefinalvirtual

Interrupt does nothing.

Implements Dispatcher.

Definition at line 221 of file dispatcher.hpp.

◆ IRecv()

MPI_Request IRecv	(	Connection &	c,
		uint32_t	seq,
		void *	data,
		size_t	size
	)

Definition at line 83 of file dispatcher.cpp.

References LOG, Connection::peer(), and Connection::rx_bytes_.

Referenced by Dispatcher::PerformAsync(), and Connection::SyncRecv().

◆ ISend()

MPI_Request ISend	(	Connection &	c,
		uint32_t	seq,
		const void *	data,
		size_t	size
	)

Definition at line 61 of file dispatcher.cpp.

References LOG, Connection::peer(), and Connection::tx_bytes_.

Referenced by Dispatcher::PerformAsync(), Connection::SyncSend(), and Connection::SyncSendRecv().

◆ PerformAsync()

void PerformAsync ( MpiAsync && a )

Issue the encapsulated request to the MPI layer.

Definition at line 190 of file dispatcher.cpp.

References AsyncReadBuffer::connection(), AsyncWriteBuffer::connection(), AsyncReadByteBlock::connection(), AsyncWriteBlock::connection(), AsyncReadBuffer::data(), AsyncWriteBuffer::data(), AsyncReadByteBlock::data(), AsyncWriteBlock::data(), Dispatcher::IRecv(), Dispatcher::ISend(), Dispatcher::mpi_async_, Dispatcher::mpi_async_out_, Dispatcher::mpi_async_requests_, Dispatcher::mpi_status_out_, Connection::peer(), Dispatcher::MpiAsync::READ_BUFFER, Dispatcher::MpiAsync::READ_BYTE_BLOCK, Dispatcher::recv_active_, Dispatcher::send_active_, AsyncReadBuffer::size(), AsyncWriteBuffer::size(), AsyncReadByteBlock::size(), AsyncWriteBlock::size(), Dispatcher::MpiAsync::WRITE_BLOCK, and Dispatcher::MpiAsync::WRITE_BUFFER.

Referenced by Dispatcher::PumpRecvQueue(), Dispatcher::PumpSendQueue(), Dispatcher::QueueAsyncRecv(), and Dispatcher::QueueAsyncSend().

◆ PumpRecvQueue()

void PumpRecvQueue ( int peer )

Check recv queue and perform waiting requests.

Definition at line 174 of file dispatcher.cpp.

References LOG, Dispatcher::PerformAsync(), Dispatcher::recv_active_, Dispatcher::recv_queue_, and tlx::unused().

Referenced by Dispatcher::DispatchOne().

◆ PumpSendQueue()

void PumpSendQueue ( int peer )

Check send queue and perform waiting requests.

Definition at line 158 of file dispatcher.cpp.

References LOG, Dispatcher::PerformAsync(), Dispatcher::send_active_, Dispatcher::send_queue_, and tlx::unused().

Referenced by Dispatcher::DispatchOne().

◆ QueueAsyncRecv()

void QueueAsyncRecv	(	net::Connection &	c,
		MpiAsync &&	a
	)

Enqueue and run the encapsulated result.

Definition at line 137 of file dispatcher.cpp.

References Connection::peer(), Dispatcher::PerformAsync(), Dispatcher::recv_active_, Dispatcher::recv_queue_, and tlx::unused().

◆ QueueAsyncSend()

void QueueAsyncSend	(	net::Connection &	c,
		MpiAsync &&	a
	)

Enqueue and run the encapsulated result.

Definition at line 117 of file dispatcher.cpp.

References Connection::peer(), Dispatcher::PerformAsync(), Dispatcher::send_active_, Dispatcher::send_queue_, and tlx::unused().

◆ SetExcept()

void SetExcept	(	net::Connection &	c,
		const Callback &	except_cb
	)

inline

Register a buffered write callback and a default exception callback.

Definition at line 113 of file dispatcher.hpp.

References Connection::peer().

Member Data Documentation

◆ debug

constexpr bool debug = false

staticprivate

Definition at line 81 of file dispatcher.hpp.

◆ mpi_async_

std::vector<MpiAsync> mpi_async_

private

array of asynchronous writers and readers (these have to align with mpi_async_requests_)

Definition at line 415 of file dispatcher.hpp.

Referenced by Dispatcher::AddAsyncRequest(), Dispatcher::DispatchOne(), Dispatcher::PerformAsync(), and Dispatcher::~Dispatcher().

◆ mpi_async_out_

std::vector<int> mpi_async_out_

private

array of output integer of finished requests for MPI_Testsome().

Definition at line 421 of file dispatcher.hpp.

Referenced by Dispatcher::AddAsyncRequest(), Dispatcher::DispatchOne(), and Dispatcher::PerformAsync().

◆ mpi_async_requests_

std::vector<MPI_Request> mpi_async_requests_

private

array of current async MPI_Request for MPI_Testsome().

Definition at line 418 of file dispatcher.hpp.

Referenced by Dispatcher::AddAsyncRequest(), Dispatcher::DispatchOne(), Dispatcher::PerformAsync(), and Dispatcher::~Dispatcher().

◆ mpi_status_out_

std::vector<MPI_Status> mpi_status_out_

private

array of output MPI_Status of finished requests for MPI_Testsome().

Definition at line 424 of file dispatcher.hpp.

Referenced by Dispatcher::AddAsyncRequest(), Dispatcher::DispatchOne(), and Dispatcher::PerformAsync().

◆ recv_active_

std::vector<size_t> recv_active_

private

number of active requests

Definition at line 437 of file dispatcher.hpp.

Referenced by Dispatcher::Dispatcher(), Dispatcher::DispatchOne(), Dispatcher::PerformAsync(), Dispatcher::PumpRecvQueue(), and Dispatcher::QueueAsyncRecv().

◆ recv_queue_

std::deque<std::deque<MpiAsync> > recv_queue_

private

queue of delayed requests for each peer

Definition at line 431 of file dispatcher.hpp.

Referenced by Dispatcher::Dispatcher(), Dispatcher::PumpRecvQueue(), and Dispatcher::QueueAsyncRecv().

◆ send_active_

std::vector<size_t> send_active_

private

number of active requests

Definition at line 434 of file dispatcher.hpp.

Referenced by Dispatcher::Dispatcher(), Dispatcher::DispatchOne(), Dispatcher::PerformAsync(), Dispatcher::PumpSendQueue(), and Dispatcher::QueueAsyncSend().

◆ send_queue_

std::deque<std::deque<MpiAsync> > send_queue_

private

queue of delayed requests for each peer

Definition at line 428 of file dispatcher.hpp.

Referenced by Dispatcher::Dispatcher(), Dispatcher::PumpSendQueue(), and Dispatcher::QueueAsyncSend().

◆ watch_

std::vector<Watch> watch_

private

callback watch vector

Definition at line 236 of file dispatcher.hpp.

Referenced by Dispatcher::Dispatcher(), and Dispatcher::DispatchOne().

◆ watch_active_

size_t watch_active_ { 0 }

private

counter of active watches

Definition at line 239 of file dispatcher.hpp.

Referenced by Dispatcher::DispatchOne().

The documentation for this class was generated from the following files:

thrill/net/mpi/dispatcher.hpp
thrill/net/mpi/dispatcher.cpp

Detailed Description

Classes

Public Types

Public Member Functions

Static Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Member Typedef Documentation

◆ Callback

Constructor & Destructor Documentation

◆ Dispatcher()

◆ ~Dispatcher()

Member Function Documentation

◆ AddAsyncRequest()

◆ AddRead()

◆ AddWrite()

◆ AsyncRead() [1/2]

◆ AsyncRead() [2/2]

◆ AsyncWrite() [1/2]

◆ AsyncWrite() [2/2]

◆ Cancel()

◆ DefaultExceptionCallback()

◆ DispatchOne()

◆ Interrupt()

◆ IRecv()

◆ ISend()

◆ PerformAsync()

◆ PumpRecvQueue()

◆ PumpSendQueue()

◆ QueueAsyncRecv()

◆ QueueAsyncSend()

◆ SetExcept()

Member Data Documentation

◆ debug

◆ mpi_async_

◆ mpi_async_out_

◆ mpi_async_requests_

◆ mpi_status_out_

◆ recv_active_

◆ recv_queue_

◆ send_active_

◆ send_queue_

◆ watch_

◆ watch_active_