Thrill  0.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
UnionNode< ValueType > Class Template Referencefinal

Detailed Description

template<typename ValueType>
class thrill::api::UnionNode< ValueType >

Implements a Union operation by hooking any number of input DIANodes and forwarding the output immediately to all children.

The problem with Union is that children can be added to the node after creation (as with all other nodes). This however requires the UnionNode to remember which of its children has already got which input's items. This is recorded in each UnionChild's pushed_inputs array.

For example this occurs in the following DIA graph:

[Gen.1] ---—v [Union.3] –> [Sort.4] ----—> [Size.6] +—^ +----------------------------—> [Size.7] [Gen.2] –+ +---------------------—> [Size.5]

Size.5 triggers Execute and PushData such that Sort.4's PreOp gets all data from Gen.2, but not from Gen.1. Then, running Size.6 requires Union.3 to get data from Gen.1 and NOT run Gen.2 again.

Another situation then occur, when [Size.7] is added later.

Definition at line 53 of file union.hpp.

+ Inheritance diagram for UnionNode< ValueType >:
+ Collaboration diagram for UnionNode< ValueType >:

#include <union.hpp>

Classes

struct  UnionChild
 

Public Types

using Callback = typename Super::Callback
 
enum  ChildStatus { NEW, PUSHING, DONE }
 
using Super = DIANode< ValueType >
 
- Public Types inherited from DIANode< ValueType >
using Callback = tlx::delegate< void(const ValueType &)>
 
- Public Types inherited from DIABase
using DIABasePtr = tlx::CountingPtr< DIABase >
 

Public Member Functions

template<typename ParentDIA0 , typename... ParentDIAs>
 UnionNode (const ParentDIA0 &parent0, const ParentDIAs &...parents)
 
void AddChild (DIABase *node, const Callback &callback, size_t parent_index=0) final
 Enables children to push their "folded" function chains to their parent. More...
 
std::vector< DIABase * > children () const final
 Returns the children of this DIABase. More...
 
size_t consume_counter () const final
 Returns consume_counter_. More...
 
void DecConsumeCounter (size_t consume) final
 
void Execute () final
 Virtual execution method. Triggers actual computation in sub-classes. More...
 
bool ForwardDataOnly () const final
 A UnionNode cannot be executed, it never contains any data. More...
 
void IncConsumeCounter (size_t consume) final
 
void PushData (bool) final
 Virtual method for pushing data. Triggers actual pushing in sub-classes. More...
 
void PushItem (const ValueType &item, size_t parent_index) const
 Method for derived classes to Push a single item to all children. More...
 
void RemoveAllChildren () final
 
void RemoveChild (DIABase *node) final
 
bool RequireParentPushData (size_t parent_index) const final
 Check whether we need PushData() from the specific parent. More...
 
void RunPushData () final
 
void SetConsumeCounter (size_t consume) final
 
void StartPreOp (size_t id) final
 Virtual method for preparing start of PushData. More...
 
void StopPreOp (size_t id) final
 Virtual method for preparing end of PushData. More...
 
- Public Member Functions inherited from DIANode< ValueType >
 DIANode (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
 Constructor for a DIANode, which sets references to the parent nodes. More...
 
 DIANode (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
 Constructor for a DIANode, which sets references to the parent nodes. More...
 
void PushFile (data::File &file, bool consume) const
 
void PushItem (const ValueType &item) const
 Method for derived classes to Push a single item to all children. More...
 
- Public Member Functions inherited from DIABase
 DIABase (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
 The constructor for a DIABase. More...
 
 DIABase (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
 The constructor for a DIABase. More...
 
 DIABase (const DIABase &)=delete
 non-copyable: delete copy-constructor More...
 
 DIABase (DIABase &&)=default
 move-constructor: default More...
 
virtual ~DIABase ()
 Virtual destructor for a DIABase. More...
 
Contextcontext ()
 Returns the api::Context of this DIABase. More...
 
const size_t & id () const
 return unique id() of DIANode subclass as stored by StatsNode More...
 
const char * label () const
 return label() of DIANode subclass as stored by StatsNode More...
 
mem::Managermem_manager ()
 Return the Context's memory manager. More...
 
DIABaseoperator= (const DIABase &)=delete
 non-copyable: delete assignment operator More...
 
DIABaseoperator= (DIABase &&)=default
 move-assignment operator: default More...
 
std::vector< size_t > parent_ids () const
 Returns the parents of this DIABase. More...
 
const std::vector< DIABasePtr > & parents () const
 Returns the parents of this DIABase. More...
 
void RemoveParent (DIABase *p)
 Remove a parent. More...
 
void RunScope ()
 
void set_mem_limit (const DIAMemUse &mem_limit)
 
void set_state (const DIAState &state)
 
DIAState state () const
 
virtual DIAMemUse PreOpMemUse ()
 Amount of RAM used by PreOp after StartPreOp() More...
 
virtual bool OnPreOpFile (const data::File &, size_t)
 
virtual DIAMemUse ExecuteMemUse ()
 Amount of RAM used by Execute() More...
 
virtual DIAMemUse PushDataMemUse ()
 Amount of RAM used by PushData() More...
 
virtual void Dispose ()
 Virtual clear method. Triggers actual disposing in sub-classes. More...
 
- Public Member Functions inherited from ReferenceCounter
 ReferenceCounter () noexcept
 new objects have zero reference count More...
 
 ReferenceCounter (const ReferenceCounter &) noexcept
 coping still creates a new object with zero reference count More...
 
 ~ReferenceCounter ()
 
bool dec_reference () const noexcept
 Call whenever resetting (i.e. More...
 
void inc_reference () const noexcept
 Call whenever setting a pointer to the object. More...
 
ReferenceCounteroperator= (const ReferenceCounter &) noexcept
 assignment operator, leaves pointers unchanged More...
 
size_t reference_count () const noexcept
 Return the number of references to this object (for debugging) More...
 
bool unique () const noexcept
 Test if the ReferenceCounter is referenced by only one CountingPtr. More...
 

Private Attributes

std::vector< UnionChildchildren_
 Callback functions from the child nodes. More...
 
size_t num_inputs_
 

Static Private Attributes

static constexpr bool debug = false
 

Additional Inherited Members

- Public Attributes inherited from DIABase
common::JsonLogger logger_
 
- Static Public Attributes inherited from DIABase
static constexpr size_t kNeverConsume = static_cast<size_t>(-1)
 Never full consume. More...
 
- Protected Attributes inherited from DIANode< ValueType >
std::vector< Childchildren_
 Callback functions from the child nodes. More...
 
- Protected Attributes inherited from DIABase
Contextcontext_
 associated Context More...
 
const size_t id_
 DIA serial id. More...
 
const char *const label_
 DOp node static label. More...
 
DIAState state_ = DIAState::NEW
 State of the DIANode. State is NEW on creation. More...
 
std::vector< DIABasePtrparents_
 Parents of this DIABase. More...
 
DIAMemUse mem_limit_ = 0
 
size_t consume_counter_ = 1
 

Member Typedef Documentation

using Callback = typename Super::Callback

Definition at line 60 of file union.hpp.

using Super = DIANode<ValueType>

Definition at line 58 of file union.hpp.

Member Enumeration Documentation

enum ChildStatus
strong
Enumerator
NEW 
PUSHING 
DONE 

Definition at line 62 of file union.hpp.

Constructor & Destructor Documentation

UnionNode ( const ParentDIA0 &  parent0,
const ParentDIAs &...  parents 
)
inlineexplicit

Constructor for variant with variadic parent parameter pack, which each parent may have a different FunctionStack.

Definition at line 89 of file union.hpp.

Member Function Documentation

void AddChild ( DIABase node,
const Callback callback,
size_t  parent_index = 0 
)
inlinefinalvirtual

Enables children to push their "folded" function chains to their parent.

This way the parent can push all its result elements to each of the children. This procedure enables the minimization of IO-accesses.

Reimplemented from DIANode< ValueType >.

Definition at line 163 of file union.hpp.

References UnionNode< ValueType >::children_, UnionNode< ValueType >::NEW, and UnionNode< ValueType >::num_inputs_.

std::vector<DIABase*> children ( ) const
inlinefinalvirtual

Returns the children of this DIABase.

Reimplemented from DIANode< ValueType >.

Definition at line 207 of file union.hpp.

References UnionNode< ValueType >::children_.

size_t consume_counter ( ) const
inlinefinalvirtual

Returns consume_counter_.

Reimplemented from DIABase.

Definition at line 269 of file union.hpp.

References DIABase::kNeverConsume, min(), and DIABase::parents_.

void DecConsumeCounter ( size_t  counter)
inlinefinalvirtual

Virtual SetConsume flag which is called by the user via .Keep() or .Consume() to set consumption.

Reimplemented from DIABase.

Definition at line 285 of file union.hpp.

References DIABase::parents_.

void Execute ( )
inlinefinalvirtual

Virtual execution method. Triggers actual computation in sub-classes.

Implements DIABase.

Definition at line 226 of file union.hpp.

bool ForwardDataOnly ( ) const
inlinefinalvirtual

A UnionNode cannot be executed, it never contains any data.

Reimplemented from DIABase.

Definition at line 216 of file union.hpp.

void IncConsumeCounter ( size_t  counter)
inlinefinalvirtual

Virtual SetConsume flag which is called by the user via .Keep() or .Consume() to set consumption.

Reimplemented from DIABase.

Definition at line 278 of file union.hpp.

References DIABase::parents_.

void PushData ( bool  consume)
inlinefinalvirtual

Virtual method for pushing data. Triggers actual pushing in sub-classes.

Implements DIABase.

Definition at line 267 of file union.hpp.

void PushItem ( const ValueType &  item,
size_t  parent_index 
) const
inline

Method for derived classes to Push a single item to all children.

Definition at line 242 of file union.hpp.

References UnionNode< ValueType >::children_.

void RemoveAllChildren ( )
inlinefinalvirtual

Virtual method for removing all childs. Triggers actual removing in sub-classes.

Reimplemented from DIANode< ValueType >.

Definition at line 183 of file union.hpp.

References UnionNode< ValueType >::children_, and UnionNode< ValueType >::DONE.

void RemoveChild ( DIABase node)
inlinefinalvirtual

Remove a child from the vector of children. This method is called by the destructor of children.

Reimplemented from DIANode< ValueType >.

Definition at line 173 of file union.hpp.

References UnionNode< ValueType >::children_.

bool RequireParentPushData ( size_t  parent_index) const
inlinefinalvirtual

Check whether we need PushData() from the specific parent.

Reimplemented from DIABase.

Definition at line 219 of file union.hpp.

References UnionNode< ValueType >::children_.

void RunPushData ( )
inlinefinalvirtual

Performing push operation. Notifies children and calls actual push method. Then cleans up the DIA graph by freeing parent references of children.

Reimplemented from DIANode< ValueType >.

Definition at line 265 of file union.hpp.

void SetConsumeCounter ( size_t  counter)
inlinefinalvirtual

Virtual SetConsume flag which is called by the user via .Keep() or .Consume() to set consumption.

Reimplemented from DIABase.

Definition at line 292 of file union.hpp.

References DIABase::parents_.

void StartPreOp ( size_t  )
inlinefinalvirtual

Virtual method for preparing start of PushData.

Reimplemented from DIABase.

Definition at line 228 of file union.hpp.

References UnionNode< ValueType >::children_, DIABase::id(), LOG, LOG0, UnionNode< ValueType >::NEW, and UnionNode< ValueType >::PUSHING.

void StopPreOp ( size_t  )
inlinefinalvirtual

Virtual method for preparing end of PushData.

Reimplemented from DIABase.

Definition at line 249 of file union.hpp.

References UnionNode< ValueType >::children_, UnionNode< ValueType >::DONE, DIABase::id(), LOG, LOG0, and UnionNode< ValueType >::PUSHING.

Member Data Documentation

constexpr bool debug = false
staticprivate

Definition at line 55 of file union.hpp.

size_t num_inputs_
private

Definition at line 300 of file union.hpp.

Referenced by UnionNode< ValueType >::AddChild().


The documentation for this class was generated from the following file: