Detailed Description

template<typename ValueType>
class thrill::api::DIANode< ValueType >

A DIANode is a typed node representing and operation in Thrill.

It is the super class for all operation nodes and stores the state of the operation.

Template Parameters

ValueType The output type of the DIA delivered by this DIANode.

Definition at line 37 of file dia_node.hpp.

Inheritance diagram for DIANode< ValueType >:

Collaboration diagram for DIANode< ValueType >:

#include <dia_node.hpp>

Classes
struct	Child

Public Types
using	Callback = tlx::delegate< void(const ValueType &)>

Public Types inherited from DIABase
using	DIABasePtr = tlx::CountingPtr< DIABase >

Public Member Functions
	DIANode (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
	Constructor for a DIANode, which sets references to the parent nodes. More...

	DIANode (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
	Constructor for a DIANode, which sets references to the parent nodes. More...

virtual void	AddChild (DIABase *node, const Callback &callback=Callback(), size_t parent_index=0)
	Enables children to push their "folded" function chains to their parent. More...

std::vector< DIABase * >	children () const override
	Returns the children of this DIABase. More...

void	PushFile (data::File &file, bool consume) const

void	PushItem (const ValueType &item) const
	Method for derived classes to Push a single item to all children. More...

void	RemoveAllChildren () override

void	RemoveChild (DIABase *node) override

void	RunPushData () override

Public Member Functions inherited from DIABase
	DIABase (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
	The constructor for a DIABase. More...

	DIABase (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
	The constructor for a DIABase. More...

	DIABase (const DIABase &)=delete
	non-copyable: delete copy-constructor More...

	DIABase (DIABase &&)=default
	move-constructor: default More...

virtual	~DIABase ()
	Virtual destructor for a DIABase. More...

virtual size_t	consume_counter () const
	Returns consume_counter_. More...

Context &	context ()
	Returns the api::Context of this DIABase. More...

virtual void	DecConsumeCounter (size_t counter)

const size_t &	dia_id () const
	return unique id of DIANode subclass as stored by StatsNode More...

virtual bool	ForwardDataOnly () const

virtual void	IncConsumeCounter (size_t counter)

const char *	label () const
	return label() of DIANode subclass as stored by StatsNode More...

mem::Manager &	mem_manager ()
	Return the Context's memory manager. More...

DIABase &	operator= (const DIABase &)=delete
	non-copyable: delete assignment operator More...

DIABase &	operator= (DIABase &&)=default
	move-assignment operator: default More...

std::vector< size_t >	parent_ids () const
	Returns the parents of this DIABase. More...

const std::vector< DIABasePtr > &	parents () const
	Returns the parents of this DIABase. More...

void	RemoveParent (DIABase *p)
	Remove a parent. More...

virtual bool	RequireParentPushData (size_t) const

void	RunScope ()

void	set_mem_limit (const DIAMemUse &mem_limit)

void	set_state (const DIAState &state)

virtual void	SetConsumeCounter (size_t counter)

DIAState	state () const

virtual DIAMemUse	PreOpMemUse ()
	Amount of RAM used by PreOp after StartPreOp() More...

virtual void	StartPreOp (size_t)
	Virtual method for preparing start of PushData. More...

virtual bool	OnPreOpFile (const data::File &, size_t)

virtual void	StopPreOp (size_t)
	Virtual method for preparing end of PushData. More...

virtual DIAMemUse	ExecuteMemUse ()
	Amount of RAM used by Execute() More...

virtual void	Execute ()=0
	Virtual execution method. Triggers actual computation in sub-classes. More...

virtual DIAMemUse	PushDataMemUse ()
	Amount of RAM used by PushData() More...

virtual void	PushData (bool consume)=0
	Virtual method for pushing data. Triggers actual pushing in sub-classes. More...

virtual void	Dispose ()
	Virtual clear method. Triggers actual disposing in sub-classes. More...

Public Member Functions inherited from ReferenceCounter
	ReferenceCounter () noexcept
	new objects have zero reference count More...

	ReferenceCounter (const ReferenceCounter &) noexcept
	coping still creates a new object with zero reference count More...

	~ReferenceCounter ()

bool	dec_reference () const noexcept
	Call whenever resetting (i.e. More...

void	inc_reference () const noexcept
	Call whenever setting a pointer to the object. More...

ReferenceCounter &	operator= (const ReferenceCounter &) noexcept
	assignment operator, leaves pointers unchanged More...

size_t	reference_count () const noexcept
	Return the number of references to this object (for debugging) More...

bool	unique () const noexcept
	Test if the ReferenceCounter is referenced by only one CountingPtr. More...

Protected Attributes
std::vector< Child >	children_
	Callback functions from the child nodes. More...

Protected Attributes inherited from DIABase
Context &	context_
	associated Context More...

const size_t	dia_id_
	DIA serial id. More...

const char *const	label_
	DOp node static label. More...

DIAState	state_ = DIAState::NEW
	State of the DIANode. State is NEW on creation. More...

std::vector< DIABasePtr >	parents_
	Parents of this DIABase. More...

DIAMemUse	mem_limit_ = 0

size_t	consume_counter_ = 1

Additional Inherited Members
Public Attributes inherited from DIABase
common::JsonLogger	logger_

Static Public Attributes inherited from DIABase
static constexpr size_t	kNeverConsume = static_cast<size_t>(-1)
	Never full consume. More...

Member Typedef Documentation

◆ Callback

using Callback = tlx::delegate<void (const ValueType&)>

Definition at line 40 of file dia_node.hpp.

Constructor & Destructor Documentation

◆ DIANode() [1/2]

DIANode	(	Context &	ctx,
		const char *	label,
		const std::initializer_list< size_t > &	parent_ids,
		const std::initializer_list< DIABasePtr > &	parents
	)

inline

Constructor for a DIANode, which sets references to the parent nodes.

Calls the constructor of DIABase with the same parameters.

Definition at line 57 of file dia_node.hpp.

◆ DIANode() [2/2]

DIANode	(	Context &	ctx,
		const char *	label,
		std::vector< size_t > &&	parent_ids,
		std::vector< DIABasePtr > &&	parents
	)

inline

Constructor for a DIANode, which sets references to the parent nodes.

Calls the constructor of DIABase with the same parameters.

Definition at line 66 of file dia_node.hpp.

Member Function Documentation

◆ AddChild()

virtual void AddChild	(	DIABase *	node,
		const Callback &	callback = `Callback()`,
		size_t	parent_index = `0`
	)

inlinevirtual

Enables children to push their "folded" function chains to their parent.

This way the parent can push all its result elements to each of the children. This procedure enables the minimization of IO-accesses.

Reimplemented in UnionNode< ValueType >.

Definition at line 76 of file dia_node.hpp.

Referenced by ReduceNode< ValueType, KeyExtractor, ReduceFunction, ReduceConfig, KeyHashFunction, KeyEqualFunction, VolatileKey, UseDuplicateDetection >::ReduceNode(), and ReduceToIndexNode< ValueType, KeyExtractor, ReduceFunction, ReduceConfig, VolatileKey, SkipPreReducePhase >::ReduceToIndexNode().

◆ children()

std::vector<DIABase*> children ( ) const

inlineoverridevirtual

Returns the children of this DIABase.

Implements DIABase.

Reimplemented in UnionNode< ValueType >.

Definition at line 112 of file dia_node.hpp.

◆ PushFile()

void PushFile	(	data::File &	file,
		bool	consume
	)		const

inline

Method for derived classes to Push a whole File of ValueType items to all children.

Definition at line 156 of file dia_node.hpp.

Referenced by CollapseNode< ValueType >::OnPreOpFile(), CacheNode< ValueType >::PushData(), SortNode< ValueType, CompareFunction, SortAlgorithm, Stable >::PushData(), and ReadBinaryNode< ValueType >::PushData().

◆ PushItem()

void PushItem ( const ValueType & item ) const

inline

Method for derived classes to Push a single item to all children.

Definition at line 147 of file dia_node.hpp.

Referenced by CollapseNode< ValueType >::CollapseNode(), JoinNode< ValueType, FirstDIA, SecondDIA, KeyExtractor1, KeyExtractor2, JoinFunction, HashFunction, UseLocationDetection >::JoinAllElements(), EqualToDIANode< ValueType >::PushData(), ConcatToDIANode< ValueType >::PushData(), GenerateNode< ValueType, GenerateFunction >::PushData(), DistributeNode< ValueType >::PushData(), PrefixSumNode< ValueType, SumFunction, Inclusive >::PushData(), ZipWithIndexNode< ValueType, ZipFunction >::PushData(), RebalanceNode< ValueType >::PushData(), GroupToIndexNode< ValueType, KeyExtractor, GroupFunction >::PushData(), SampleNode< ValueType >::PushData(), ZipNode< ValueType, ZipFunction, Pad, UnequalCheck, NoRebalance, kNumInputs >::PushData(), MergeNode< ValueType, Comparator, kNumInputs >::PushData(), OverlapWindowNode< ValueType, Input, WindowFunction, PartialWindowFunction >::PushData(), GroupByNode< ValueType, KeyExtractor, GroupFunction, HashFunction, UseLocationDetection >::PushData(), SortNode< ValueType, CompareFunction, SortAlgorithm, Stable >::PushData(), ReadBinaryNode< ValueType >::PushData(), ConcatNode< ValueType >::PushData(), DisjointWindowNode< ValueType, Input, WindowFunction, PartialWindowFunction >::PushData(), GroupToIndexNode< ValueType, KeyExtractor, GroupFunction >::RunUserFunc(), and GroupByNode< ValueType, KeyExtractor, GroupFunction, HashFunction, UseLocationDetection >::RunUserFunc().

◆ RemoveAllChildren()

void RemoveAllChildren ( )

inlineoverridevirtual

Virtual method for removing all childs. Triggers actual removing in sub-classes.

Implements DIABase.

Reimplemented in UnionNode< ValueType >.

Definition at line 93 of file dia_node.hpp.

◆ RemoveChild()

void RemoveChild ( DIABase * node )

inlineoverridevirtual

Remove a child from the vector of children. This method is called by the destructor of children.

Implements DIABase.

Reimplemented in UnionNode< ValueType >.

Definition at line 83 of file dia_node.hpp.

◆ RunPushData()

void RunPushData ( )

inlineoverridevirtual

Performing push operation. Notifies children and calls actual push method. Then cleans up the DIA graph by freeing parent references of children.

Implements DIABase.

Reimplemented in UnionNode< ValueType >.

Definition at line 123 of file dia_node.hpp.

Member Data Documentation

◆ children_

std::vector<Child> children_

protected

Callback functions from the child nodes.

Definition at line 181 of file dia_node.hpp.

Referenced by DIANode< StackInput >::AddChild(), DIANode< StackInput >::children(), DIANode< StackInput >::PushFile(), DIANode< StackInput >::PushItem(), DIANode< StackInput >::RemoveAllChildren(), DIANode< StackInput >::RemoveChild(), DIANode< StackInput >::RunPushData(), CollapseNode< ValueType >::StartPreOp(), and CollapseNode< ValueType >::StopPreOp().

The documentation for this class was generated from the following file:

thrill/api/dia_node.hpp

Detailed Description

template<typename ValueType> class thrill::api::DIANode< ValueType >

Classes

Public Types

Public Member Functions

Protected Attributes

Additional Inherited Members

Member Typedef Documentation

◆ Callback

Constructor & Destructor Documentation

◆ DIANode() [1/2]

◆ DIANode() [2/2]

Member Function Documentation

◆ AddChild()

◆ children()

◆ PushFile()

◆ PushItem()

◆ RemoveAllChildren()

◆ RemoveChild()

◆ RunPushData()

Member Data Documentation

◆ children_

template<typename ValueType>
class thrill::api::DIANode< ValueType >