Thrill  0.1
ZipWindowNode< ValueType, ZipFunction_, Pad_, UnequalCheck, UseStdArray, kNumInputs_ > Class Template Referencefinal

Detailed Description

template<typename ValueType, typename ZipFunction_, bool Pad_, bool UnequalCheck, bool UseStdArray, size_t kNumInputs_>
class thrill::api::ZipWindowNode< ValueType, ZipFunction_, Pad_, UnequalCheck, UseStdArray, kNumInputs_ >

Definition at line 175 of file zip_window.hpp.

+ Inheritance diagram for ZipWindowNode< ValueType, ZipFunction_, Pad_, UnequalCheck, UseStdArray, kNumInputs_ >:
+ Collaboration diagram for ZipWindowNode< ValueType, ZipFunction_, Pad_, UnequalCheck, UseStdArray, kNumInputs_ >:

#include <zip_window.hpp>

Classes

class  RegisterParent
 Register Parent PreOp Hooks, instantiated and called for each Zip parent. More...
 

Public Types

template<size_t Index>
using ZipArgN = typename ZipWindowTraits< ZipFunction >::template value_type_plain< Index >
 
using ZipArgsTuple = typename ZipWindowTraits< ZipFunction >::value_type_tuple_plain
 
using ZipFunction = ZipFunction_
 
- Public Types inherited from DOpNode< ValueType >
using Super = DIANode< ValueType >
 
- Public Types inherited from DIANode< ValueType >
using Callback = tlx::delegate< void(const ValueType &)>
 
- Public Types inherited from DIABase
using DIABasePtr = tlx::CountingPtr< DIABase >
 

Public Member Functions

template<typename ParentDIA0 , typename... ParentDIAs>
 ZipWindowNode (const std::array< size_t, kNumInputs > &window_size, const ZipFunction &zip_function, const ZipArgsTuple &padding, const ParentDIA0 &parent0, const ParentDIAs &... parents)
 Constructor for a ZipWindowNode. More...
 
void Dispose () final
 Virtual clear method. Triggers actual disposing in sub-classes. More...
 
void Execute () final
 Virtual execution method. Triggers actual computation in sub-classes. More...
 
bool OnPreOpFile (const data::File &file, size_t parent_index) final
 Receive a whole data::File of ValueType, but only if our stack is empty. More...
 
void PushData (bool consume) final
 Virtual method for pushing data. Triggers actual pushing in sub-classes. More...
 
void StartPreOp (size_t parent_index) final
 Virtual method for preparing start of PushData. More...
 
void StopPreOp (size_t parent_index) final
 Virtual method for preparing end of PushData. More...
 
- Public Member Functions inherited from DOpNode< ValueType >
 DOpNode (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
 Constructor for a DOpNode, which sets references to the parent nodes. More...
 
 DOpNode (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
 Constructor for a DOpNode, which sets references to the parent nodes. More...
 
- Public Member Functions inherited from DIANode< ValueType >
 DIANode (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
 Constructor for a DIANode, which sets references to the parent nodes. More...
 
 DIANode (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
 Constructor for a DIANode, which sets references to the parent nodes. More...
 
virtual void AddChild (DIABase *node, const Callback &callback=Callback(), size_t parent_index=0)
 Enables children to push their "folded" function chains to their parent. More...
 
std::vector< DIABase * > children () const override
 Returns the children of this DIABase. More...
 
void PushFile (data::File &file, bool consume) const
 
void PushItem (const ValueType &item) const
 Method for derived classes to Push a single item to all children. More...
 
void RemoveAllChildren () override
 
void RemoveChild (DIABase *node) override
 
void RunPushData () override
 
- Public Member Functions inherited from DIABase
 DIABase (Context &ctx, const char *label, const std::initializer_list< size_t > &parent_ids, const std::initializer_list< DIABasePtr > &parents)
 The constructor for a DIABase. More...
 
 DIABase (Context &ctx, const char *label, std::vector< size_t > &&parent_ids, std::vector< DIABasePtr > &&parents)
 The constructor for a DIABase. More...
 
 DIABase (const DIABase &)=delete
 non-copyable: delete copy-constructor More...
 
 DIABase (DIABase &&)=default
 move-constructor: default More...
 
virtual ~DIABase ()
 Virtual destructor for a DIABase. More...
 
virtual size_t consume_counter () const
 Returns consume_counter_. More...
 
Contextcontext ()
 Returns the api::Context of this DIABase. More...
 
virtual void DecConsumeCounter (size_t counter)
 
const size_t & dia_id () const
 return unique id of DIANode subclass as stored by StatsNode More...
 
virtual bool ForwardDataOnly () const
 
virtual void IncConsumeCounter (size_t counter)
 
const char * label () const
 return label() of DIANode subclass as stored by StatsNode More...
 
mem::Managermem_manager ()
 Return the Context's memory manager. More...
 
DIABaseoperator= (const DIABase &)=delete
 non-copyable: delete assignment operator More...
 
DIABaseoperator= (DIABase &&)=default
 move-assignment operator: default More...
 
std::vector< size_t > parent_ids () const
 Returns the parents of this DIABase. More...
 
const std::vector< DIABasePtr > & parents () const
 Returns the parents of this DIABase. More...
 
void RemoveParent (DIABase *p)
 Remove a parent. More...
 
virtual bool RequireParentPushData (size_t) const
 
void RunScope ()
 
void set_mem_limit (const DIAMemUse &mem_limit)
 
void set_state (const DIAState &state)
 
virtual void SetConsumeCounter (size_t counter)
 
DIAState state () const
 
virtual DIAMemUse PreOpMemUse ()
 Amount of RAM used by PreOp after StartPreOp() More...
 
virtual DIAMemUse ExecuteMemUse ()
 Amount of RAM used by Execute() More...
 
virtual DIAMemUse PushDataMemUse ()
 Amount of RAM used by PushData() More...
 
- Public Member Functions inherited from ReferenceCounter
 ReferenceCounter () noexcept
 new objects have zero reference count More...
 
 ReferenceCounter (const ReferenceCounter &) noexcept
 coping still creates a new object with zero reference count More...
 
 ~ReferenceCounter ()
 
bool dec_reference () const noexcept
 Call whenever resetting (i.e. More...
 
void inc_reference () const noexcept
 Call whenever setting a pointer to the object. More...
 
ReferenceCounteroperator= (const ReferenceCounter &) noexcept
 assignment operator, leaves pointers unchanged More...
 
size_t reference_count () const noexcept
 Return the number of references to this object (for debugging) More...
 
bool unique () const noexcept
 Test if the ReferenceCounter is referenced by only one CountingPtr. More...
 

Static Public Attributes

static constexpr size_t kNumInputs = kNumInputs_
 
static constexpr bool Pad = Pad_
 
- Static Public Attributes inherited from DIABase
static constexpr size_t kNeverConsume = static_cast<size_t>(-1)
 Never full consume. More...
 

Private Types

using Super = DOpNode< ValueType >
 

Private Member Functions

template<size_t Index>
void DoScatter ()
 Scatter items from DIA "Index" to other workers if necessary. More...
 
void MainOp ()
 Receive elements from other workers. More...
 

Private Attributes

std::vector< data::Filefiles_
 Files for intermediate storage. More...
 
const ZipArgsTuple padding_
 padding for shorter DIAs More...
 
const std::array< bool, kNumInputsparent_stack_empty_
 Whether the parent stack is empty. More...
 
data::CatStreamPtr streams_ [kNumInputs]
 Array of inbound CatStreams. More...
 
const std::array< size_t, kNumInputswindow_size_
 Size k of the windows. More...
 
data::File::Writer writers_ [kNumInputs]
 Writers to intermediate files. More...
 
ZipFunction zip_function_
 Zip function. More...
 
Variables for Calculating Exchange
std::array< size_t, kNumInputssize_prefixsum_
 exclusive prefix sum over the number of items in workers More...
 
size_t result_window_count_
 shortest size of Zipped inputs More...
 

Static Private Attributes

static constexpr bool debug = false
 
static constexpr bool stats_enabled = false
 Set this variable to true to enable generation and output of stats. More...
 

Additional Inherited Members

- Public Attributes inherited from DIABase
common::JsonLogger logger_
 
- Protected Attributes inherited from DIANode< ValueType >
std::vector< Childchildren_
 Callback functions from the child nodes. More...
 
- Protected Attributes inherited from DIABase
Contextcontext_
 associated Context More...
 
const size_t dia_id_
 DIA serial id. More...
 
const char *const label_
 DOp node static label. More...
 
DIAState state_ = DIAState::NEW
 State of the DIANode. State is NEW on creation. More...
 
std::vector< DIABasePtrparents_
 Parents of this DIABase. More...
 
DIAMemUse mem_limit_ = 0
 
size_t consume_counter_ = 1
 

Member Typedef Documentation

◆ Super

using Super = DOpNode<ValueType>
private

Definition at line 182 of file zip_window.hpp.

◆ ZipArgN

using ZipArgN = typename ZipWindowTraits<ZipFunction>::template value_type_plain<Index>

Definition at line 192 of file zip_window.hpp.

◆ ZipArgsTuple

using ZipArgsTuple = typename ZipWindowTraits<ZipFunction>::value_type_tuple_plain

Definition at line 194 of file zip_window.hpp.

◆ ZipFunction

using ZipFunction = ZipFunction_

Definition at line 186 of file zip_window.hpp.

Constructor & Destructor Documentation

◆ ZipWindowNode()

ZipWindowNode ( const std::array< size_t, kNumInputs > &  window_size,
const ZipFunction zip_function,
const ZipArgsTuple padding,
const ParentDIA0 &  parent0,
const ParentDIAs &...  parents 
)
inline

Constructor for a ZipWindowNode.

Definition at line 201 of file zip_window.hpp.

References tlx::call_foreach_with_index(), and thrill::mem::padding.

Member Function Documentation

◆ Dispose()

void Dispose ( )
inlinefinalvirtual

Virtual clear method. Triggers actual disposing in sub-classes.

Reimplemented from DIABase.

Definition at line 281 of file zip_window.hpp.

◆ DoScatter()

void DoScatter ( )
inlineprivate

Scatter items from DIA "Index" to other workers if necessary.

Definition at line 357 of file zip_window.hpp.

References LOG, and min().

◆ Execute()

void Execute ( )
inlinefinalvirtual

Virtual execution method. Triggers actual computation in sub-classes.

Implements DIABase.

Definition at line 252 of file zip_window.hpp.

◆ MainOp()

void MainOp ( )
inlineprivate

Receive elements from other workers.

Definition at line 394 of file zip_window.hpp.

References die, sLOG, tlx::unused(), and thrill::common::VecToStr().

◆ OnPreOpFile()

bool OnPreOpFile ( const data::File file,
size_t  parent_index 
)
inlinefinalvirtual

Receive a whole data::File of ValueType, but only if our stack is empty.

Reimplemented from DIABase.

Definition at line 230 of file zip_window.hpp.

References LOGC.

◆ PushData()

void PushData ( bool  consume)
inlinefinalvirtual

Virtual method for pushing data. Triggers actual pushing in sub-classes.

Implements DIABase.

Definition at line 256 of file zip_window.hpp.

References tlx::apply_tuple().

◆ StartPreOp()

void StartPreOp ( size_t  )
inlinefinalvirtual

Virtual method for preparing start of PushData.

Reimplemented from DIABase.

Definition at line 225 of file zip_window.hpp.

◆ StopPreOp()

void StopPreOp ( size_t  )
inlinefinalvirtual

Virtual method for preparing end of PushData.

Reimplemented from DIABase.

Definition at line 247 of file zip_window.hpp.

References LOG.

Member Data Documentation

◆ debug

constexpr bool debug = false
staticprivate

Definition at line 177 of file zip_window.hpp.

◆ files_

std::vector<data::File> files_
private

Files for intermediate storage.

Definition at line 301 of file zip_window.hpp.

◆ kNumInputs

constexpr size_t kNumInputs = kNumInputs_
static

Definition at line 188 of file zip_window.hpp.

◆ Pad

constexpr bool Pad = Pad_
static

Definition at line 187 of file zip_window.hpp.

◆ padding_

const ZipArgsTuple padding_
private

padding for shorter DIAs

Definition at line 295 of file zip_window.hpp.

◆ parent_stack_empty_

const std::array<bool, kNumInputs> parent_stack_empty_
private

Whether the parent stack is empty.

Definition at line 298 of file zip_window.hpp.

◆ result_window_count_

size_t result_window_count_
private

shortest size of Zipped inputs

Definition at line 316 of file zip_window.hpp.

◆ size_prefixsum_

std::array<size_t, kNumInputs> size_prefixsum_
private

exclusive prefix sum over the number of items in workers

Definition at line 313 of file zip_window.hpp.

◆ stats_enabled

constexpr bool stats_enabled = false
staticprivate

Set this variable to true to enable generation and output of stats.

Definition at line 180 of file zip_window.hpp.

◆ streams_

data::CatStreamPtr streams_[kNumInputs]
private

Array of inbound CatStreams.

Definition at line 307 of file zip_window.hpp.

◆ window_size_

const std::array<size_t, kNumInputs> window_size_
private

Size k of the windows.

Definition at line 289 of file zip_window.hpp.

◆ writers_

data::File::Writer writers_[kNumInputs]
private

Writers to intermediate files.

Definition at line 304 of file zip_window.hpp.

◆ zip_function_

ZipFunction zip_function_
private

Zip function.

Definition at line 292 of file zip_window.hpp.


The documentation for this class was generated from the following file: