Thrill  0.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
HostContext Class Reference

Detailed Description

The HostContext contains all data structures shared among workers on the same host.

It is used to construct and destroy them. For testing multiple instances are run in the same process.

Definition at line 87 of file context.hpp.

+ Collaboration diagram for HostContext:

#include <context.hpp>

Public Member Functions

 HostContext (size_t local_host_id, const MemoryConfig &mem_config, std::unique_ptr< net::DispatcherThread > dispatcher, std::array< net::GroupPtr, net::Manager::kGroupCount > &&groups, size_t workers_per_host)
 constructor from existing net Groups. Used by the construction methods. More...
 
 ~HostContext ()
 destructor More...
 
data::BlockPoolblock_pool ()
 the block manager keeps all data blocks moving through the system. More...
 
data::Multiplexerdata_multiplexer ()
 data multiplexer transmits large amounts of data asynchronously. More...
 
net::FlowControlChannelManagerflow_manager ()
 the flow control group is used for collective communication. More...
 
size_t host_rank () const
 
size_t local_host_id () const
 Returns local_host_id_. More...
 
std::string MakeHostLogPath (size_t host_rank)
 create host log More...
 
MemoryConfigmem_config ()
 host-global memory config More...
 
mem::Managermem_manager ()
 host-global memory manager More...
 
net::Managernet_manager ()
 net manager constructs communication groups to other hosts. More...
 
size_t worker_mem_limit () const
 memory limit of each worker Context for local data structures More...
 
size_t workers_per_host () const
 number of workers per host (all have the same). More...
 

Static Public Member Functions

static std::vector
< std::unique_ptr< HostContext > > 
ConstructLoopback (size_t num_hosts, size_t workers_per_host)
 Construct a number of mock hosts running in this process. More...
 

Public Attributes

Logging System
common::JsonLogger base_logger_
 base logger exclusive for this host context More...
 
common::JsonLogger logger_
 
std::unique_ptr
< common::ProfileThread
profiler_
 thread for scheduling profiling methods for statistical output More...
 

Private Attributes

data::BlockPool block_pool_
 data block pool More...
 
common::ProfileTaskRegistration block_pool_profiler_
 register BlockPool's profiling method More...
 
data::Multiplexer data_multiplexer_
 data multiplexer transmits large amounts of data asynchronously. More...
 
std::unique_ptr
< net::DispatcherThread
dispatcher_
 main host network dispatcher thread backend More...
 
net::FlowControlChannelManager flow_manager_
 the flow control group is used for collective communication. More...
 
size_t local_host_id_
 id among all local hosts (in test program runs) More...
 
MemoryConfig mem_config_
 memory configuration More...
 
mem::Manager mem_manager_ { nullptr, "HostContext" }
 host-global memory manager for internal memory only More...
 
net::Manager net_manager_
 net manager constructs communication groups to other hosts. More...
 
common::ProfileTaskRegistration net_manager_profiler_
 register net_manager_'s profiling method More...
 
size_t workers_per_host_
 number of workers per host (all have the same). More...
 

Constructor & Destructor Documentation

HostContext ( size_t  local_host_id,
const MemoryConfig mem_config,
std::unique_ptr< net::DispatcherThread dispatcher,
std::array< net::GroupPtr, net::Manager::kGroupCount > &&  groups,
size_t  workers_per_host 
)

constructor from existing net Groups. Used by the construction methods.

Definition at line 1059 of file context.cpp.

References thrill::common::LogCmdlineParams(), HostContext::logger_, HostContext::profiler_, thrill::common::StartLinuxProcStatsProfiler(), and thrill::mem::StartMemProfiler().

destructor

Definition at line 1084 of file context.cpp.

References HostContext::dispatcher_.

Member Function Documentation

data::BlockPool& block_pool ( )
inline

the block manager keeps all data blocks moving through the system.

Definition at line 136 of file context.hpp.

References HostContext::block_pool_.

std::vector< std::unique_ptr< HostContext > > ConstructLoopback ( size_t  num_hosts,
size_t  workers_per_host 
)
static

Construct a number of mock hosts running in this process.

Definition at line 306 of file context.cpp.

References HostContext::mem_config(), MemoryConfig::setup(), MemoryConfig::verbose_, and HostContext::workers_per_host().

data::Multiplexer& data_multiplexer ( )
inline

data multiplexer transmits large amounts of data asynchronously.

Definition at line 139 of file context.hpp.

References HostContext::data_multiplexer_.

net::FlowControlChannelManager& flow_manager ( )
inline

the flow control group is used for collective communication.

Definition at line 133 of file context.hpp.

References HostContext::flow_manager_.

size_t host_rank ( ) const
inline

Returns id of this host in the cluser. A host is a machine in the cluster that hosts multiple workers

Definition at line 130 of file context.hpp.

References Manager::my_host_rank(), and HostContext::net_manager_.

size_t local_host_id ( ) const
inline

Returns local_host_id_.

Definition at line 109 of file context.hpp.

References HostContext::local_host_id_.

std::string MakeHostLogPath ( size_t  host_rank)

create host log

Definition at line 1089 of file context.cpp.

References HostContext::mem_config(), and thrill::mem::to_string().

MemoryConfig& mem_config ( )
inline

host-global memory config

Definition at line 120 of file context.hpp.

References HostContext::mem_config_.

Referenced by HostContext::ConstructLoopback(), and HostContext::MakeHostLogPath().

mem::Manager& mem_manager ( )
inline

host-global memory manager

Definition at line 123 of file context.hpp.

References HostContext::mem_manager_.

net::Manager& net_manager ( )
inline

net manager constructs communication groups to other hosts.

Definition at line 126 of file context.hpp.

References HostContext::net_manager_.

size_t worker_mem_limit ( ) const
inline

memory limit of each worker Context for local data structures

Definition at line 115 of file context.hpp.

References HostContext::mem_config_, MemoryConfig::ram_workers_, and HostContext::workers_per_host_.

size_t workers_per_host ( ) const
inline

number of workers per host (all have the same).

Definition at line 112 of file context.hpp.

References HostContext::workers_per_host_.

Referenced by HostContext::ConstructLoopback().

Member Data Documentation

common::JsonLogger base_logger_

base logger exclusive for this host context

Definition at line 150 of file context.hpp.

data::BlockPool block_pool_
private
Initial value:

data block pool

Definition at line 191 of file context.hpp.

Referenced by HostContext::block_pool().

common::ProfileTaskRegistration block_pool_profiler_
private
Initial value:
{
std::chrono::milliseconds(500), *profiler_, &block_pool_
}

register BlockPool's profiling method

Definition at line 198 of file context.hpp.

data::Multiplexer data_multiplexer_
private
Initial value:

data multiplexer transmits large amounts of data asynchronously.

Definition at line 204 of file context.hpp.

Referenced by HostContext::data_multiplexer().

std::unique_ptr<net::DispatcherThread> dispatcher_
private

main host network dispatcher thread backend

Definition at line 173 of file context.hpp.

Referenced by HostContext::~HostContext().

net::FlowControlChannelManager flow_manager_
private
Initial value:

the flow control group is used for collective communication.

Definition at line 186 of file context.hpp.

Referenced by HostContext::flow_manager().

size_t local_host_id_
private

id among all local hosts (in test program runs)

Definition at line 164 of file context.hpp.

Referenced by HostContext::local_host_id().

public member which delivers key:value pairs as JSON log lines. this logger is local to this Context which is exclusive for one worker thread.

Definition at line 155 of file context.hpp.

Referenced by HostContext::HostContext().

MemoryConfig mem_config_
private

memory configuration

Definition at line 143 of file context.hpp.

Referenced by HostContext::mem_config(), and HostContext::worker_mem_limit().

mem::Manager mem_manager_ { nullptr, "HostContext" }
private

host-global memory manager for internal memory only

Definition at line 170 of file context.hpp.

Referenced by HostContext::mem_manager().

net::Manager net_manager_
private

net manager constructs communication groups to other hosts.

Definition at line 176 of file context.hpp.

Referenced by HostContext::host_rank(), and HostContext::net_manager().

common::ProfileTaskRegistration net_manager_profiler_
private
Initial value:
{
std::chrono::milliseconds(500), *profiler_, &net_manager_
}

register net_manager_'s profiling method

Definition at line 180 of file context.hpp.

std::unique_ptr<common::ProfileThread> profiler_

thread for scheduling profiling methods for statistical output

Definition at line 158 of file context.hpp.

Referenced by HostContext::HostContext().

size_t workers_per_host_
private

number of workers per host (all have the same).

Definition at line 167 of file context.hpp.

Referenced by HostContext::worker_mem_limit(), and HostContext::workers_per_host().


The documentation for this class was generated from the following files: