Detailed Description

Collection of NetConnections to workers, allows point-to-point client communication and simple collectives like MPI.

Definition at line 43 of file group.hpp.

Inheritance diagram for Group:

Collaboration diagram for Group:

#include <group.hpp>

Public Member Functions
	Group (const Group &)=delete
	non-copyable: delete copy-constructor More...

	Group (Group &&)=default
	move-constructor: default More...

Group &	operator= (const Group &)=delete
	non-copyable: delete assignment operator More...

Group &	operator= (Group &&)=default
	move-assignment operator: default More...

Public Member Functions inherited from Group
	Group (size_t my_rank)
	initializing constructor More...

	Group (const Group &)=delete
	non-copyable: delete copy-constructor More...

	Group (Group &&)=default
	move-constructor: default More...

template<>
void	AllReduce (int &value, std::plus< int >)

template<>
void	AllReduce (int &value, common::minimum< int >)

template<>
void	AllReduce (int &value, common::maximum< int >)

template<>
void	AllReduce (unsigned int &value, std::plus< unsigned int >)

template<>
void	AllReduce (unsigned int &value, common::minimum< unsigned int >)

template<>
void	AllReduce (unsigned int &value, common::maximum< unsigned int >)

template<>
void	AllReduce (long &value, std::plus< long >)

template<>
void	AllReduce (long &value, common::minimum< long >)

template<>
void	AllReduce (long &value, common::maximum< long >)

template<>
void	AllReduce (unsigned long &value, std::plus< unsigned long >)

template<>
void	AllReduce (unsigned long &value, common::minimum< unsigned long >)

template<>
void	AllReduce (unsigned long &value, common::maximum< unsigned long >)

template<>
void	AllReduce (long long &value, std::plus< long long >)

template<>
void	AllReduce (long long &value, common::minimum< long long >)

template<>
void	AllReduce (long long &value, common::maximum< long long >)

template<>
void	AllReduce (unsigned long long &value, std::plus< unsigned long long >)

template<>
void	AllReduce (unsigned long long &value, common::minimum< unsigned long long >)

template<>
void	AllReduce (unsigned long long &value, common::maximum< unsigned long long >)

template<>
void	Broadcast (int &value, size_t origin)

template<>
void	Broadcast (unsigned int &value, size_t origin)

template<>
void	Broadcast (long &value, size_t origin)

template<>
void	Broadcast (unsigned long &value, size_t origin)

template<>
void	Broadcast (long long &value, size_t origin)

template<>
void	Broadcast (unsigned long long &value, size_t origin)

template<>
void	ExPrefixSum (int &value, std::plus< int >, const int &initial)

template<>
void	ExPrefixSum (unsigned int &value, std::plus< unsigned int >, const unsigned int &initial)

template<>
void	ExPrefixSum (long &value, std::plus< long >, const long &initial)

template<>
void	ExPrefixSum (unsigned long &value, std::plus< unsigned long >, const unsigned long &initial)

template<>
void	ExPrefixSum (long long &value, std::plus< long long >, const long long &initial)

template<>
void	ExPrefixSum (unsigned long long &value, std::plus< unsigned long long >, const unsigned long long &initial)

Group &	operator= (const Group &)=delete
	non-copyable: delete assignment operator More...

Group &	operator= (Group &&)=default
	move-assignment operator: default More...

template<>
void	PrefixSum (int &value, std::plus< int >, const int &initial)

template<>
void	PrefixSum (unsigned int &value, std::plus< unsigned int >, const unsigned int &initial)

template<>
void	PrefixSum (long &value, std::plus< long >, const long &initial)

template<>
void	PrefixSum (unsigned long &value, std::plus< unsigned long >, const unsigned long &initial)

template<>
void	PrefixSum (long long &value, std::plus< long long >, const long long &initial)

template<>
void	PrefixSum (unsigned long long &value, std::plus< unsigned long long >, const unsigned long long &initial)

size_t	my_host_rank () const
	Return our rank among hosts in this group. More...

virtual size_t	num_parallel_async () const

size_t	OneFactorSize () const
	Number of of 1-factor iterations. More...

size_t	OneFactorPeer (size_t round) const

template<typename T >
void	SendTo (size_t dest, const T &data)
	Sends a serializable type to the given peer. More...

template<typename T >
void	ReceiveFrom (size_t src, T *data)
	Receives a serializable type from the given peer. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	PrefixSum (T &value, BinarySumOp sum_op=BinarySumOp(), const T &initial=T())
	Calculate inclusive prefix sum. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	ExPrefixSum (T &value, BinarySumOp sum_op=BinarySumOp(), const T &initial=T())
	Calculate exclusive prefix sum. More...

template<typename T >
void	Broadcast (T &value, size_t origin=0)
	Broadcast a value from the worker "origin". More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	Reduce (T &value, size_t root=0, BinarySumOp sum_op=BinarySumOp())
	Reduce a value from all workers to the worker 0. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	AllReduce (T &value, BinarySumOp sum_op=BinarySumOp())
	Reduce a value from all workers to all workers. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	PrefixSumSelect (T &value, BinarySumOp sum_op=BinarySumOp(), const T &initial=T(), bool inclusive=true)
	select prefixsum implementation (often due to total number of processors) More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	PrefixSumDoubling (T &value, BinarySumOp sum_op=BinarySumOp(), const T &initial=T(), bool inclusive=true)
	Calculate for every worker his prefix sum. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	PrefixSumHypercube (T &value, BinarySumOp sum_op=BinarySumOp())
	Calculate for every worker his prefix sum. More...

template<typename T >
void	BroadcastSelect (T &value, size_t origin=0)
	select broadcast implementation (often due to total number of processors) More...

template<typename T >
void	BroadcastTrivial (T &value, size_t origin=0)
	Broadcasts the value of the peer with index 0 to all the others. More...

template<typename T >
void	BroadcastBinomialTree (T &value, size_t origin=0)
	Broadcasts the value of the worker with index "origin" to all the others. More...

template<typename T >
void	AllGatherRecursiveDoublingPowerOfTwo (T *values, size_t n)

template<typename T >
void	AllGatherBruck (T *values, size_t n)

template<typename T , typename BinarySumOp = std::plus<T>>
void	AllReduceSelect (T &value, BinarySumOp sum_op=BinarySumOp())
	select allreduce implementation (often due to total number of processors) More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	AllReduceSimple (T &value, BinarySumOp sum_op=BinarySumOp())
	Perform an All-Reduce on the workers. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	AllReduceAtRoot (T &value, BinarySumOp sum_op=BinarySumOp())
	Broadcasts the value of the peer with index 0 to all the others. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	AllReduceHypercube (T &value, BinarySumOp sum_op=BinarySumOp())
	Perform an All-Reduce for powers of two. More...

template<typename T , typename BinarySumOp = std::plus<T>>
void	AllReduceElimination (T &value, BinarySumOp sum_op=BinarySumOp())
	Perform an All-Reduce using the elimination protocol described in R. More...

Private Attributes
std::vector< Connection >	connections_
	Connections to all other clients in the Group. More...

Static Private Attributes
static constexpr bool	debug = false

Construction and Initialization
static std::vector< std::unique_ptr< Group > >	ConstructLoopbackMesh (size_t num_hosts)
	Construct a test network with an underlying full mesh of local loopback stream sockets for testing. More...

static std::vector< std::unique_ptr< Group > >	ConstructLocalRealTCPMesh (size_t num_hosts)
	Construct a test network with an underlying full mesh of REAL tcp streams interconnected via localhost ports. More...

	Group (size_t my_rank, size_t group_size)
	Initializing constructor, used by tests for creating Groups. More...

Status and Access to NetConnections
using	Dispatcher = tcp::SelectDispatcher

Connection &	tcp_connection (size_t id)
	Return Connection to client id. More...

net::Connection &	connection (size_t id) final
	Return Connection to client id. More...

std::unique_ptr< net::Dispatcher >	ConstructDispatcher () const final

Connection &	AssignConnection (Connection &connection)
	Assigns a connection to this net group. More...

size_t	num_hosts () const final
	Return number of connections in this group (= number computing hosts) More...

void	Close ()
	Closes all client connections. More...

	~Group ()
	Closes all client connections. More...

Additional Inherited Members
Protected Member Functions inherited from Group
virtual void	PrefixSumPlusInt (int &value, const int &initial)

virtual void	ExPrefixSumPlusInt (int &value, const int &initial)

virtual void	BroadcastInt (int &value, size_t origin)

virtual void	AllReducePlusInt (int &value)

virtual void	AllReduceMinimumInt (int &value)

virtual void	AllReduceMaximumInt (int &value)

virtual void	PrefixSumPlusUnsignedInt (unsigned int &value, const unsigned int &initial)

virtual void	ExPrefixSumPlusUnsignedInt (unsigned int &value, const unsigned int &initial)

virtual void	BroadcastUnsignedInt (unsigned int &value, size_t origin)

virtual void	AllReducePlusUnsignedInt (unsigned int &value)

virtual void	AllReduceMinimumUnsignedInt (unsigned int &value)

virtual void	AllReduceMaximumUnsignedInt (unsigned int &value)

virtual void	PrefixSumPlusLong (long &value, const long &initial)

virtual void	ExPrefixSumPlusLong (long &value, const long &initial)

virtual void	BroadcastLong (long &value, size_t origin)

virtual void	AllReducePlusLong (long &value)

virtual void	AllReduceMinimumLong (long &value)

virtual void	AllReduceMaximumLong (long &value)

virtual void	PrefixSumPlusUnsignedLong (unsigned long &value, const unsigned long &initial)

virtual void	ExPrefixSumPlusUnsignedLong (unsigned long &value, const unsigned long &initial)

virtual void	BroadcastUnsignedLong (unsigned long &value, size_t origin)

virtual void	AllReducePlusUnsignedLong (unsigned long &value)

virtual void	AllReduceMinimumUnsignedLong (unsigned long &value)

virtual void	AllReduceMaximumUnsignedLong (unsigned long &value)

virtual void	PrefixSumPlusLongLong (long long &value, const long long &initial)

virtual void	ExPrefixSumPlusLongLong (long long &value, const long long &initial)

virtual void	BroadcastLongLong (long long &value, size_t origin)

virtual void	AllReducePlusLongLong (long long &value)

virtual void	AllReduceMinimumLongLong (long long &value)

virtual void	AllReduceMaximumLongLong (long long &value)

virtual void	PrefixSumPlusUnsignedLongLong (unsigned long long &value, const unsigned long long &initial)

virtual void	ExPrefixSumPlusUnsignedLongLong (unsigned long long &value, const unsigned long long &initial)

virtual void	BroadcastUnsignedLongLong (unsigned long long &value, size_t origin)

virtual void	AllReducePlusUnsignedLongLong (unsigned long long &value)

virtual void	AllReduceMinimumUnsignedLongLong (unsigned long long &value)

virtual void	AllReduceMaximumUnsignedLongLong (unsigned long long &value)

template<typename T , typename BinarySumOp >
T	SendReceiveReduce (size_t peer, const T &value, BinarySumOp sum_op)
	Helper method for AllReduce(). More...

template<typename T , typename BinarySumOp >
void	AllReduceEliminationProcess (size_t host_id, size_t group_size, size_t remaining_hosts, size_t send_to, T &value, BinarySumOp sum_op)
	Helper method for AllReduce(). More...

Protected Attributes inherited from Group
size_t	my_rank_
	our rank in the network group More...

Member Typedef Documentation

◆ Dispatcher

using Dispatcher = tcp::SelectDispatcher

Definition at line 104 of file group.hpp.

Constructor & Destructor Documentation

◆ Group() [1/3]

Group	(	size_t	my_rank,
		size_t	group_size
	)

inline

Initializing constructor, used by tests for creating Groups.

Definition at line 68 of file group.hpp.

References Group::operator=().

◆ Group() [2/3]

Group ( const Group & )

delete

non-copyable: delete copy-constructor

◆ Group() [3/3]

Group ( Group && )

default

move-constructor: default

◆ ~Group()

~Group ( )

inlinevirtual

Closes all client connections.

Reimplemented from Group.

Definition at line 148 of file group.hpp.

References Group::Close().

Member Function Documentation

◆ AssignConnection()

Connection& AssignConnection ( Connection & connection )

inline

Assigns a connection to this net group.

This method swaps the net connection to memory managed by this group. The reference given to that method will be invalid afterwards.

Parameters

connection The connection to assign.

Returns: A ref to the assigned connection, which is always valid, but might be different from the inut connection.

Definition at line 118 of file group.hpp.

References Group::connections_, and thrill::mem::to_string().

◆ Close()

void Close ( )

inlinevirtual

Closes all client connections.

Implements Group.

Definition at line 135 of file group.hpp.

References Group::connections_, and Group::my_rank_.

Referenced by Group::~Group().

◆ connection()

net::Connection& connection ( size_t id )

inlinefinalvirtual

Return Connection to client id.

Implements Group.

Definition at line 100 of file group.hpp.

References Group::tcp_connection().

◆ ConstructDispatcher()

std::unique_ptr< Dispatcher > ConstructDispatcher ( ) const

finalvirtual

Construct a network dispatcher object for the network backend used by this group, matching its internal implementation. A dispatcher may be shared between groups of the same type.

Implements Group.

Definition at line 31 of file group.cpp.

◆ ConstructLocalRealTCPMesh()

std::vector< std::unique_ptr< Group > > ConstructLocalRealTCPMesh ( size_t num_hosts )

static

Construct a test network with an underlying full mesh of REAL tcp streams interconnected via localhost ports.

Definition at line 64 of file group.cpp.

References thrill::net::tcp::Construct(), Group::num_hosts(), sLOG, and thrill::mem::to_string().

◆ ConstructLoopbackMesh()

std::vector< std::unique_ptr< Group > > ConstructLoopbackMesh ( size_t num_hosts )

static

Construct a test network with an underlying full mesh of local loopback stream sockets for testing.

Returns vector of net::Group interfaces for each virtual client. This is ideal for testing network communication protocols.

Definition at line 36 of file group.cpp.

References Socket::CreatePair(), LOG, and Group::num_hosts().

Referenced by thrill::api::RunLocalSameThread(), and thrill::net::RunLoopbackGroupTest().

◆ num_hosts()

size_t num_hosts ( ) const

inlinefinalvirtual

Return number of connections in this group (= number computing hosts)

Implements Group.

Definition at line 130 of file group.hpp.

References Group::connections_.

Referenced by Group::ConstructLocalRealTCPMesh(), and Group::ConstructLoopbackMesh().

◆ operator=() [1/2]

Group& operator= ( const Group & )

delete

non-copyable: delete assignment operator

Referenced by Group::Group().

◆ operator=() [2/2]

Group& operator= ( Group && )

default

move-assignment operator: default

◆ tcp_connection()

Connection& tcp_connection ( size_t id )

inline

Return Connection to client id.

Definition at line 87 of file group.hpp.

References Group::connections_, Group::my_rank_, and thrill::mem::to_string().

Referenced by Group::connection().

Member Data Documentation

◆ connections_

std::vector<Connection> connections_

private

Connections to all other clients in the Group.

Definition at line 156 of file group.hpp.

Referenced by Group::AssignConnection(), Group::Close(), Group::num_hosts(), and Group::tcp_connection().

◆ debug

constexpr bool debug = false

staticprivate

Definition at line 45 of file group.hpp.

The documentation for this class was generated from the following files:

thrill/net/tcp/group.hpp
thrill/net/tcp/group.cpp

Detailed Description

Public Member Functions

Private Attributes

Static Private Attributes

Construction and Initialization

Status and Access to NetConnections

Additional Inherited Members

Member Typedef Documentation

◆ Dispatcher

Constructor & Destructor Documentation

◆ Group() [1/3]

◆ Group() [2/3]

◆ Group() [3/3]

◆ ~Group()

Member Function Documentation

◆ AssignConnection()

◆ Close()

◆ connection()

◆ ConstructDispatcher()

◆ ConstructLocalRealTCPMesh()

◆ ConstructLoopbackMesh()

◆ num_hosts()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ tcp_connection()

Member Data Documentation

◆ connections_

◆ debug