Thrill  0.1
Network Layer

The Thrill Network Layer consists of sublayers:

Group & Connection layer

The Group object is similar to an MPI communicator: it has connections to p peers/clients/workers in a group. The network peers have only an indirect relation to the worker concept of higher Thrill layers, though they will probably be implementing using a Group.

Group provides

  • a num_hosts() and my_host_rank() function to describe the size of the group.
  • a point-to-point Connection class with a rich set of Send, Receive methods to transfer serializable items to another peer in the group: Send<T> and Receive<T>. For more low-level packet-based communication there are further methods available.
  • collective operations for any serializable datatypes: PrefixSum.

If any operation on a Connection fails, then an Exception is thrown.

For testing purposes Group provides the function ExecuteGroupThreads which creates a mock Group of k partners. The partners are connected via internal sockets (real kernel level sockets), and for each partner a std::thread is spawned. The thread executes a test function which gets a Group object that can communicate with the other partners. This is ideal for testing network protocols. See the test-net-group.cpp for more examples.

The network layer is designed to allow plugging in new network implementations by deriving from the main classes Group, Connection, and Dispatcher. The Group represents the set of connections, and provides synchronous communication, while the Dispatcher allows asynchronous callbacks.

There are currently two network implementations:

  • a mock network, which does no real communication,
  • a TCP BSD socket implementation, which is used on Linux/FreeBSD/etc.

An important design decision of the network layer is to support only sending and receiving of whole packets. The received packet size must be known by the receiver in advance, which necessitates pre-communication of sizes. This is because RDMA works in this manner.

TCP Socket API

The BSD socket API is difficult to work with correctly, which is why the following classes provide convenience functions and excessive logging facilities.