Thrill
0.1

Classes  
class  JoinNode< ValueType, FirstDIA, SecondDIA, KeyExtractor1, KeyExtractor2, JoinFunction, HashFunction, UseLocationDetection > 
Performs an inner join between two DIAs. More...  
Functions  
template<typename FirstDIA , typename... DIAs>  
auto  Concat (const FirstDIA &first_dia, const DIAs &...dias) 
Concat is a DOp, which concatenates any number of DIAs to a single DIA. More...  
template<typename ValueType >  
auto  Concat (const std::initializer_list< DIA< ValueType > > &dias) 
Concat is a DOp, which concatenates any number of DIAs to a single DIA. More...  
template<typename ValueType >  
auto  Concat (const std::vector< DIA< ValueType > > &dias) 
Concat is a DOp, which concatenates any number of DIAs to a single DIA. More...  
template<bool LocationDetectionValue, typename FirstDIA , typename SecondDIA , typename KeyExtractor1 , typename KeyExtractor2 , typename JoinFunction , typename HashFunction = std::hash<typename common::FunctionTraits<KeyExtractor1>::result_type>>  
auto  InnerJoin (const LocationDetectionFlag< LocationDetectionValue > &, const FirstDIA &first_dia, const SecondDIA &second_dia, const KeyExtractor1 &key_extractor1, const KeyExtractor2 &key_extractor2, const JoinFunction &join_function, const HashFunction &hash_function=HashFunction()) 
Performs an inner join between this DIA and the DIA given in the first parameter. More...  
template<typename FirstDIA , typename SecondDIA , typename KeyExtractor1 , typename KeyExtractor2 , typename JoinFunction , typename HashFunction = std::hash<typename common::FunctionTraits<KeyExtractor1>::result_type>>  
auto  InnerJoin (const FirstDIA &first_dia, const SecondDIA &second_dia, const KeyExtractor1 &key_extractor1, const KeyExtractor2 &key_extractor2, const JoinFunction &join_function, const HashFunction &hash_function=HashFunction()) 
Performs an inner join between this DIA and the DIA given in the first parameter. More...  
template<typename Comparator , typename FirstDIA , typename... DIAs>  
auto  Merge (const Comparator &comparator, const FirstDIA &first_dia, const DIAs &...dias) 
Merge is a DOp, which merges any number of sorted DIAs to a single sorted DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  Zip (const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  Zip (struct CutTag, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips any number of DIAs of equal size in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  Zip (struct PadTag, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips any number of DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  Zip (struct PadTag, const ZipFunction &zip_function, const typename common::FunctionTraits< ZipFunction >::args_tuple_plain &padding, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips any number of DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  Zip (struct NoRebalanceTag, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips any number of DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  ZipWindow (const std::array< size_t, 1+sizeof...(DIAs)> &window_size, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  ZipWindow (struct CutTag, const std::array< size_t, 1+sizeof...(DIAs)> &window_size, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  ZipWindow (struct PadTag, const std::array< size_t, 1+sizeof...(DIAs)> &window_size, const ZipFunction &zip_function, const typename ZipWindowTraits< ZipFunction >::value_type_tuple_plain &padding, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  ZipWindow (struct PadTag, const std::array< size_t, 1+sizeof...(DIAs)> &window_size, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  ZipWindow (struct ArrayTag, struct PadTag, const std::array< size_t, 1+sizeof...(DIAs)> &window_size, const ZipFunction &zip_function, const typename ZipWindowTraits< ZipFunction >::value_type_tuple_plain &padding, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename FirstDIAType , typename FirstDIAStack , typename... DIAs>  
auto  ZipWindow (struct ArrayTag, struct PadTag, const std::array< size_t, 1+sizeof...(DIAs)> &window_size, const ZipFunction &zip_function, const DIA< FirstDIAType, FirstDIAStack > &first_dia, const DIAs &...dias) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA. More...  
Distributed Operations (DOps)  
template<typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceConfig>  
auto  ReduceByKey (const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config=ReduceConfig()) const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename KeyExtractor , typename ReduceFunction , typename ReduceConfig , typename KeyHashFunction >  
auto  ReduceByKey (const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config, const KeyHashFunction &key_hash_function) const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename KeyExtractor , typename ReduceFunction , typename ReduceConfig , typename KeyHashFunction , typename KeyEqualFunction >  
auto  ReduceByKey (const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config, const KeyHashFunction &key_hash_function, const KeyEqualFunction &key_equal_function) const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function. More...  
template<bool VolatileKeyValue, typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceConfig, typename KeyHashFunction = std::hash<typename FunctionTraits<KeyExtractor>::result_type>, typename KeyEqualFunction = std::equal_to<typename FunctionTraits<KeyExtractor>::result_type>>  
auto  ReduceByKey (const VolatileKeyFlag< VolatileKeyValue > &, const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config=ReduceConfig(), const KeyHashFunction &key_hash_function=KeyHashFunction(), const KeyEqualFunction &key_equal_function=KeyEqualFunction()) const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function. More...  
template<bool DuplicateDetectionValue, typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceConfig, typename KeyHashFunction = std::hash<typename FunctionTraits<KeyExtractor>::result_type>, typename KeyEqualFunction = std::equal_to<typename FunctionTraits<KeyExtractor>::result_type>>  
auto  ReduceByKey (const DuplicateDetectionFlag< DuplicateDetectionValue > &, const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config=ReduceConfig(), const KeyHashFunction &key_hash_function=KeyHashFunction(), const KeyEqualFunction &key_equal_function=KeyEqualFunction()) const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function. More...  
template<bool VolatileKeyValue, bool DuplicateDetectionValue, typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceConfig, typename KeyHashFunction = std::hash<typename FunctionTraits<KeyExtractor>::result_type>, typename KeyEqualFunction = std::equal_to<typename FunctionTraits<KeyExtractor>::result_type>>  
auto  ReduceByKey (const VolatileKeyFlag< VolatileKeyValue > &, const DuplicateDetectionFlag< DuplicateDetectionValue > &, const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config=ReduceConfig(), const KeyHashFunction &key_hash_function=KeyHashFunction(), const KeyEqualFunction &key_equal_function=KeyEqualFunction()) const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename ReduceFunction , typename ReduceConfig = class DefaultReduceConfig>  
auto  ReducePair (const ReduceFunction &reduce_function, const ReduceConfig &reduce_config=ReduceConfig()) const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename ReduceFunction , typename ReduceConfig , typename KeyHashFunction >  
auto  ReducePair (const ReduceFunction &reduce_function, const ReduceConfig &reduce_config, const KeyHashFunction &key_hash_function) const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename ReduceFunction , typename ReduceConfig , typename KeyHashFunction , typename KeyEqualFunction >  
auto  ReducePair (const ReduceFunction &reduce_function, const ReduceConfig &reduce_config, const KeyHashFunction &key_hash_function, const KeyEqualFunction &key_equal_function) const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function. More...  
template<bool DuplicateDetectionValue, typename ReduceFunction , typename ReduceConfig = class DefaultReduceConfig, typename KeyHashFunction , typename KeyEqualFunction >  
auto  ReducePair (const DuplicateDetectionFlag< DuplicateDetectionValue > &, const ReduceFunction &reduce_function, const ReduceConfig &reduce_config=ReduceConfig(), const KeyHashFunction &key_hash_function=KeyHashFunction(), const KeyEqualFunction &key_equal_function=KeyEqualFunction()) const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceToIndexConfig>  
auto  ReduceToIndex (const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, size_t size, const ValueType &neutral_element=ValueType(), const ReduceConfig &reduce_config=ReduceConfig()) const 
ReduceToIndex is a DOp, which groups elements of the DIA with the key_extractor returning an unsigned integers and reduces each keybucket to a single element using the associative reduce_function. More...  
template<bool VolatileKeyValue, typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceToIndexConfig>  
auto  ReduceToIndex (const VolatileKeyFlag< VolatileKeyValue > &, const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, size_t size, const ValueType &neutral_element=ValueType(), const ReduceConfig &reduce_config=ReduceConfig()) const 
ReduceToIndex is a DOp, which groups elements of the DIA with the key_extractor returning an unsigned integers and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename KeyExtractor , typename ReduceFunction , typename ReduceConfig = class DefaultReduceToIndexConfig>  
auto  ReduceToIndex (const struct SkipPreReducePhaseTag &, const KeyExtractor &key_extractor, const ReduceFunction &reduce_function, size_t size, const ValueType &neutral_element=ValueType(), const ReduceConfig &reduce_config=ReduceConfig()) const 
ReduceToIndex is a DOp, which groups elements of the DIA with the key_extractor returning an unsigned integers and reduces each keybucket to a single element using the associative reduce_function. More...  
template<typename ValueOut , typename KeyExtractor , typename GroupByFunction >  
auto  GroupByKey (const KeyExtractor &key_extractor, const GroupByFunction &groupby_function) const 
GroupByKey is a DOp, which groups elements of the DIA by its key. More...  
template<typename ValueOut , typename KeyExtractor , typename GroupByFunction , typename HashFunction >  
auto  GroupByKey (const KeyExtractor &key_extractor, const GroupByFunction &groupby_function, const HashFunction &hash_function) const 
GroupByKey is a DOp, which groups elements of the DIA by its key. More...  
template<typename ValueOut , bool LocationDetectionTagValue, typename KeyExtractor , typename GroupByFunction , typename HashFunction = std::hash<typename FunctionTraits<KeyExtractor>::result_type>>  
auto  GroupByKey (const LocationDetectionFlag< LocationDetectionTagValue > &, const KeyExtractor &key_extractor, const GroupByFunction &groupby_function, const HashFunction &hash_function=HashFunction()) const 
GroupByKey is a DOp, which groups elements of the DIA by its key. More...  
template<typename ValueOut , typename KeyExtractor , typename GroupByFunction >  
auto  GroupToIndex (const KeyExtractor &key_extractor, const GroupByFunction &groupby_function, const size_t size, const ValueOut &neutral_element=ValueOut()) const 
GroupBy is a DOp, which groups elements of the DIA by its key. More...  
template<typename ZipFunction , typename SecondDIA >  
auto  Zip (const SecondDIA &second_dia, const ZipFunction &zip_function) const 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename SecondDIA >  
auto  Zip (struct CutTag const &, const SecondDIA &second_dia, const ZipFunction &zip_function) const 
Zips two DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename SecondDIA >  
auto  Zip (struct PadTag const &, const SecondDIA &second_dia, const ZipFunction &zip_function) const 
Zips two DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction , typename SecondDIA >  
auto  Zip (struct NoRebalanceTag const &, const SecondDIA &second_dia, const ZipFunction &zip_function) const 
Zips two DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA. More...  
template<typename ZipFunction >  
auto  ZipWithIndex (const ZipFunction &zip_function) const 
Zips each item of a DIA with its zerobased array index. More...  
template<typename CompareFunction = std::less<ValueType>>  
auto  Sort (const CompareFunction &compare_function=CompareFunction()) const 
Sort is a DOp, which sorts a given DIA according to the given compare_function. More...  
template<typename CompareFunction , typename SortFunction >  
auto  Sort (const CompareFunction &compare_function, const SortFunction &sort_algorithm) const 
Sort is a DOp, which sorts a given DIA according to the given compare_function. More...  
template<typename Comparator = std::less<ValueType>, typename SecondDIA >  
auto  Merge (const SecondDIA &second_dia, const Comparator &comparator=Comparator()) const 
Merge is a DOp, which merges two sorted DIAs to a single sorted DIA. More...  
template<typename SumFunction = std::plus<ValueType>>  
auto  PrefixSum (const SumFunction &sum_function=SumFunction(), const ValueType &initial_element=ValueType()) const 
PrefixSum is a DOp, which computes the (inclusive) prefix sum of all elements. More...  
template<typename SumFunction = std::plus<ValueType>>  
auto  ExPrefixSum (const SumFunction &sum_function=SumFunction(), const ValueType &initial_element=ValueType()) const 
ExPrefixSum is a DOp, which computes the exclusive prefix sum of all elements. More...  
template<typename WindowFunction >  
auto  Window (size_t window_size, const WindowFunction &window_function=WindowFunction()) const 
Window is a DOp, which applies a window function to every k consecutive items in a DIA. More...  
template<typename WindowFunction , typename PartialWindowFunction >  
auto  Window (size_t window_size, const WindowFunction &window_function, const PartialWindowFunction &partial_window_function) const 
Window is a DOp, which applies a window function to every k consecutive items in a DIA. More...  
template<typename WindowFunction >  
auto  Window (struct DisjointTag const &, size_t window_size, const WindowFunction &window_function) const 
Window is a DOp, which applies a window function to every k consecutive items in a DIA. More...  
template<typename ValueOut , typename WindowFunction >  
auto  FlatWindow (size_t window_size, const WindowFunction &window_function=WindowFunction()) const 
FlatWindow is a DOp, which applies a window function to every k consecutive items in a DIA. More...  
template<typename ValueOut , typename WindowFunction , typename PartialWindowFunction >  
auto  FlatWindow (size_t window_size, const WindowFunction &window_function, const PartialWindowFunction &partial_window_function) const 
FlatWindow is a DOp, which applies a window function to every k consecutive items in a DIA. More...  
template<typename ValueOut , typename WindowFunction >  
auto  FlatWindow (struct DisjointTag const &, size_t window_size, const WindowFunction &window_function) const 
FlatWindow is a DOp, which applies a window function to every k consecutive items in a DIA. More...  
template<typename SecondDIA >  
auto  Concat (const SecondDIA &second_dia) const 
Concat is a DOp, which concatenates any number of DIAs to a single DIA. More...  
auto  Rebalance () const 
Rebalance is a DOp, which rebalances a single DIA among all workers; in general, this operation is needed only if previous steps are known to create heavy imbalance (e.g. More...  
DIA< ValueType >  Collapse () const 
Create a CollapseNode which is mainly used to collapse the LOp chain into a DIA<T> with an empty stack. More...  
DIA< ValueType >  Cache () const 
Create a CacheNode which contains all items of a DIA in calculated plain format. More...  
DIA< ValueType > Cache  (  )  const 
DIA< ValueType > Collapse  (  )  const 
Create a CollapseNode which is mainly used to collapse the LOp chain into a DIA<T> with an empty stack.
This is most often necessary for iterative algorithms, where a DIA<T> reference variable is updated in each iteration.
Definition at line 160 of file collapse.hpp.
References CollapseSwitch< ValueType, Stack >::MakeCollapse().
auto thrill::api::Concat  (  const FirstDIA &  first_dia, 
const DIAs &...  dias  
) 
Concat is a DOp, which concatenates any number of DIAs to a single DIA.
All input DIAs must contain the same type, which is also the output DIA's type.
The concat operation balances all input data, so that each worker will have an equal number of elements when the concat completes.
Definition at line 332 of file concat.hpp.
References tlx::vexpand().
Referenced by DIA< ValueType_, Stack_ >::Concat().
auto thrill::api::Concat  (  const std::initializer_list< DIA< ValueType > > &  dias  ) 
Concat is a DOp, which concatenates any number of DIAs to a single DIA.
All input DIAs must contain the same type, which is also the output DIA's type.
The concat operation balances all input data, so that each worker will have an equal number of elements when the concat completes.
dias  DIAs, which is concatenated. 
Definition at line 355 of file concat.hpp.
auto thrill::api::Concat  (  const std::vector< DIA< ValueType > > &  dias  ) 
Concat is a DOp, which concatenates any number of DIAs to a single DIA.
All input DIAs must contain the same type, which is also the output DIA's type.
The concat operation balances all input data, so that each worker will have an equal number of elements when the concat completes.
dias  DIAs, which is concatenated. 
Definition at line 377 of file concat.hpp.
auto Concat  (  const SecondDIA< ValueType_, Stack_ > &  second_dia  )  const 
Concat is a DOp, which concatenates any number of DIAs to a single DIA.
All input DIAs must contain the same type, which is also the output DIA's type.
The concat operation balances all input data, so that each worker will have an equal number of elements when the concat completes.
Definition at line 389 of file concat.hpp.
References thrill::api::Concat().
auto ExPrefixSum  (  const SumFunction &  sum_function = SumFunction() , 
const ValueType &  initial_element = ValueType() 

)  const 
ExPrefixSum is a DOp, which computes the exclusive prefix sum of all elements.
The sum function defines how two elements are combined to a single element.
sum_function  Sum function (any associative function). 
initial_element  Initial element of the sum function. 
Definition at line 169 of file prefixsum.hpp.
References gen_data::value.
auto FlatWindow  (  size_t  window_size, 
const WindowFunction &  window_function = WindowFunction() 

)  const 
FlatWindow is a DOp, which applies a window function to every k consecutive items in a DIA.
The window function is also given the index of the first item, and can output zero or more items via an Emitter.
window_size  the size of the delivered window. Signature: TODO(tb). 
window_function  Window function applied to each k item. 
Definition at line 272 of file window.hpp.
auto FlatWindow  (  size_t  window_size, 
const WindowFunction &  window_function,  
const PartialWindowFunction &  partial_window_function  
)  const 
FlatWindow is a DOp, which applies a window function to every k consecutive items in a DIA.
The window function is also given the index of the first item, and can output zero or more items via an Emitter.
window_size  the size of the delivered window. Signature: TODO(tb). 
window_function  Window function applied to each k item. 
partial_window_function  Window function applied to less than k items. 
Definition at line 252 of file window.hpp.
auto FlatWindow  (  struct DisjointTag const &  , 
size_t  window_size,  
const WindowFunction &  window_function  
)  const 
FlatWindow is a DOp, which applies a window function to every k consecutive items in a DIA.
The window function is also given the index of the first item, and can output zero or more items via an Emitter.
window_size  the size of the delivered window. Signature: TODO(tb). 
window_function  Window function applied to each k item. 
Definition at line 508 of file window.hpp.
auto GroupByKey  (  const KeyExtractor &  key_extractor, 
const GroupByFunction &  groupby_function  
)  const 
GroupByKey is a DOp, which groups elements of the DIA by its key.
After having grouped all elements of one key, all elements of one key will be processed according to the GroupByFunction and returns an output Contrary to Reduce, GroupBy allows usage of functions that require all elements of one key at once as GroupByFunction will be applied after all elements with the same key have been grouped. However because of this reason, the communication overhead is also higher. If possible, usage of Reduce is therefore recommended.
As GroupBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
KeyExtractor  Type of the key_extractor function. The key_extractor function is equal to a map function. 
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
GroupByFunction  Type of the groupby_function. This is a function taking an iterator for all elements of the same key as input. 
groupby_function  Reduce function, which defines how the key buckets are grouped and processed. input param: api::GroupByReader with functions HasNext() and Next() 
auto GroupByKey  (  const KeyExtractor &  key_extractor, 
const GroupByFunction &  groupby_function,  
const HashFunction &  hash_function  
)  const 
GroupByKey is a DOp, which groups elements of the DIA by its key.
After having grouped all elements of one key, all elements of one key will be processed according to the GroupByFunction and returns an output Contrary to Reduce, GroupBy allows usage of functions that require all elements of one key at once as GroupByFunction will be applied after all elements with the same key have been grouped. However because of this reason, the communication overhead is also higher. If possible, usage of Reduce is therefore recommended.
As GroupBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
KeyExtractor  Type of the key_extractor function. The key_extractor function is equal to a map function. 
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
GroupByFunction  Type of the groupby_function. This is a function taking an iterator for all elements of the same key as input. 
groupby_function  Reduce function, which defines how the key buckets are grouped and processed. input param: api::GroupByReader with functions HasNext() and Next() 
hash_function  Hash method for Keys 
auto GroupByKey  (  const LocationDetectionFlag< LocationDetectionTagValue > &  , 
const KeyExtractor &  key_extractor,  
const GroupByFunction &  groupby_function,  
const HashFunction &  hash_function = HashFunction() 

)  const 
GroupByKey is a DOp, which groups elements of the DIA by its key.
After having grouped all elements of one key, all elements of one key will be processed according to the GroupByFunction and returns an output Contrary to Reduce, GroupBy allows usage of functions that require all elements of one key at once as GroupByFunction will be applied after all elements with the same key have been grouped. However because of this reason, the communication overhead is also higher. If possible, usage of Reduce is therefore recommended.
As GroupBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
KeyExtractor  Type of the key_extractor function. The key_extractor function is equal to a map function. 
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
GroupByFunction  Type of the groupby_function. This is a function taking an iterator for all elements of the same key as input. 
groupby_function  Reduce function, which defines how the key buckets are grouped and processed. input param: api::GroupByReader with functions HasNext() and Next() 
hash_function  Hash method for Keys 
auto GroupToIndex  (  const KeyExtractor &  key_extractor, 
const GroupByFunction &  groupby_function,  
const size_t  size,  
const ValueOut &  neutral_element = ValueOut() 

)  const 
GroupBy is a DOp, which groups elements of the DIA by its key.
After having grouped all elements of one key, all elements of one key will be processed according to the GroupByFunction and returns an output Contrary to Reduce, GroupBy allows usage of functions that require all elements of one key at once as GroupByFunction will be applied after all elements with the same key have been grouped. However because of this reason, the communication overhead is also higher. If possible, usage of Reduce is therefore recommended.
In contrast to GroupBy, GroupToIndex returns a DIA in a defined order, which has the reduced element with key i in position i. As GroupBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
KeyExtractor  Type of the key_extractor function. The key_extractor function is equal to a map function. 
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
GroupByFunction  Type of the groupby_function. This is a function taking an iterator for all elements of the same key as input. 
groupby_function  Reduce function, which defines how the key buckets are grouped and processed. input param: api::GroupByReader with functions HasNext() and Next() 
size  Resulting DIA size. Consequently, the key_extractor function but always return < size for any element in the input DIA. 
neutral_element  Item value with which to start the reduction in each array cell. 
auto thrill::api::InnerJoin  (  const LocationDetectionFlag< LocationDetectionValue > &  , 
const FirstDIA &  first_dia,  
const SecondDIA &  second_dia,  
const KeyExtractor1 &  key_extractor1,  
const KeyExtractor2 &  key_extractor2,  
const JoinFunction &  join_function,  
const HashFunction &  hash_function = HashFunction() 

) 
Performs an inner join between this DIA and the DIA given in the first parameter.
The key from each DIA element is hereby extracted with a key extractor function. All pairs of elements with equal keys from both DIAs are then joined with the join function.
KeyExtractor1  Type of the key_extractor1 function. This is a function from FirstDIA::ValueType to the key type. 
KeyExtractor2  Type of the key_extractor2 function. This is a function from SecondDIA::ValueType to the key type. 
JoinFunction  Type of the join_function. This is a function from ValueType and SecondDIA::ValueType to the type of the output DIA. 
first_dia  First DIA to join. 
second_dia  Second DIA to join. 
key_extractor1  Key extractor for this DIA 
key_extractor2  Key extractor for second DIA 
join_function  Join function applied to all equal key pairs 
hash_function  If necessary a hash funtion for Key 
Definition at line 713 of file inner_join.hpp.
References gen_data::value.
Referenced by examples::triangles::CountTriangles(), thrill::api::InnerJoin(), JoinTPCH4(), and examples::page_rank::PageRankJoin().
auto thrill::api::InnerJoin  (  const FirstDIA &  first_dia, 
const SecondDIA &  second_dia,  
const KeyExtractor1 &  key_extractor1,  
const KeyExtractor2 &  key_extractor2,  
const JoinFunction &  join_function,  
const HashFunction &  hash_function = HashFunction() 

) 
Performs an inner join between this DIA and the DIA given in the first parameter.
The key from each DIA element is hereby extracted with a key extractor function. All pairs of elements with equal keys from both DIAs are then joined with the join function.
KeyExtractor1  Type of the key_extractor1 function. This is a function from FirstDIA::ValueType to the key type. 
KeyExtractor2  Type of the key_extractor2 function. This is a function from SecondDIA::ValueType to the key type. 
JoinFunction  Type of the join_function. This is a function from ValueType and SecondDIA::ValueType to the type of the output DIA. 
first_dia  First DIA to join. 
second_dia  Second DIA to join. 
key_extractor1  Key extractor for this DIA 
key_extractor2  Key extractor for second DIA 
join_function  Join function applied to all equal key pairs 
hash_function  If necessary a hash funtion for Key 
Definition at line 809 of file inner_join.hpp.
References thrill::api::InnerJoin(), and thrill::api::LocationDetectionTag.
auto thrill::api::Merge  (  const Comparator &  comparator, 
const FirstDIA &  first_dia,  
const DIAs &...  dias  
) 
Merge is a DOp, which merges any number of sorted DIAs to a single sorted DIA.
All input DIAs must be sorted conforming to the given comparator. The type of the output DIA will be the type of this DIA.
The merge operation balances all input data, so that each worker will have an equal number of elements when the merge completes.
Comparator  Comparator to specify the order of input and output. 
comparator  Comparator to specify the order of input and output. 
first_dia  first DIA 
dias  DIAs, which is merged with this DIA. 
Definition at line 673 of file merge.hpp.
References gen_data::value, and tlx::vexpand().
Referenced by DIA< ValueType_, Stack_ >::Merge().
auto Merge  (  const SecondDIA< ValueType_, Stack_ > &  second_dia, 
const Comparator &  comparator = Comparator() 

)  const 
Merge is a DOp, which merges two sorted DIAs to a single sorted DIA.
Both input DIAs must be used sorted conforming to the given comparator. The type of the output DIA will be the type of this DIA.
The merge operation balances all input data, so that each worker will have an equal number of elements when the merge completes.
comparator  Comparator to specify the order of input and output. 
second_dia  DIA, which is merged with this DIA. 
Definition at line 717 of file merge.hpp.
References thrill::api::Merge().
auto PrefixSum  (  const SumFunction &  sum_function = SumFunction() , 
const ValueType &  initial_element = ValueType() 

)  const 
PrefixSum is a DOp, which computes the (inclusive) prefix sum of all elements.
The sum function defines how two elements are combined to a single element.
sum_function  Sum function (any associative function). 
initial_element  Initial element of the sum function. 
Definition at line 135 of file prefixsum.hpp.
References gen_data::value.
auto Rebalance  (  )  const 
Rebalance is a DOp, which rebalances a single DIA among all workers; in general, this operation is needed only if previous steps are known to create heavy imbalance (e.g.
like Filter()s which cut DIAs to ranges).
Definition at line 123 of file rebalance.hpp.
auto ReduceByKey  (  const KeyExtractor &  key_extractor, 
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config = ReduceConfig() 

)  const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type.
The key of the reduced element has to be equal to the keys of the input elements. Since ReduceBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
Definition at line 244 of file reduce_by_key.hpp.
References thrill::api::NoDuplicateDetectionTag, and thrill::api::NoVolatileKeyTag.
auto ReduceByKey  (  const KeyExtractor &  key_extractor, 
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config,  
const KeyHashFunction &  key_hash_function  
)  const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type.
The key of the reduced element has to be equal to the keys of the input elements. Since ReduceBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
Definition at line 259 of file reduce_by_key.hpp.
References thrill::api::NoDuplicateDetectionTag, and thrill::api::NoVolatileKeyTag.
auto ReduceByKey  (  const KeyExtractor &  key_extractor, 
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config,  
const KeyHashFunction &  key_hash_function,  
const KeyEqualFunction &  key_equal_function  
)  const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type.
The key of the reduced element has to be equal to the keys of the input elements. Since ReduceBy is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
key_equal_function  Function to compare keys in reduce hash tables. 
auto ReduceByKey  (  const VolatileKeyFlag< VolatileKeyValue > &  volatile_key_flag, 
const KeyExtractor &  key_extractor,  
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config = ReduceConfig() , 

const KeyHashFunction &  key_hash_function = KeyHashFunction() , 

const KeyEqualFunction &  key_equal_function = KeyEqualFunction() 

)  const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type.
In contrast to ReduceBy, the reduce_function is allowed to change the key (Example: Integers with modulo function as key_extractor). Creates overhead as both key and value have to be sent in shuffle step. Since ReduceByKey is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
volatile_key_flag  tag 
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
key_equal_function  Function to compare keys in reduce hash tables. 
Definition at line 276 of file reduce_by_key.hpp.
References thrill::api::NoDuplicateDetectionTag.
auto ReduceByKey  (  const DuplicateDetectionFlag< DuplicateDetectionValue > &  duplicate_detection_flag, 
const KeyExtractor &  key_extractor,  
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config = ReduceConfig() , 

const KeyHashFunction &  key_hash_function = KeyHashFunction() , 

const KeyEqualFunction &  key_equal_function = KeyEqualFunction() 

)  const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type.
In contrast to ReduceBy, the reduce_function is allowed to change the key (Example: Integers with modulo function as key_extractor). Creates overhead as both key and value have to be sent in shuffle step. Since ReduceByKey is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
duplicate_detection_flag  tag 
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
key_equal_function  Function to compare keys in reduce hash tables. 
Definition at line 294 of file reduce_by_key.hpp.
References thrill::api::NoVolatileKeyTag.
auto ReduceByKey  (  const VolatileKeyFlag< VolatileKeyValue > &  , 
const DuplicateDetectionFlag< DuplicateDetectionValue > &  ,  
const KeyExtractor &  key_extractor,  
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config = ReduceConfig() , 

const KeyHashFunction &  key_hash_function = KeyHashFunction() , 

const KeyEqualFunction &  key_equal_function = KeyEqualFunction() 

)  const 
ReduceByKey is a DOp, which groups elements of the DIA with the key_extractor and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type.
In contrast to ReduceBy, the reduce_function is allowed to change the key (Example: Integers with modulo function as key_extractor). Creates overhead as both key and value have to be sent in shuffle step. Since ReduceByKey is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
key_equal_function  Function to compare keys in reduce hash tables. 
Definition at line 313 of file reduce_by_key.hpp.
References gen_data::value.
auto ReducePair  (  const ReduceFunction &  reduce_function, 
const ReduceConfig &  reduce_config = ReduceConfig() 

)  const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type. The reduce_function is allowed to change the key. Since ReducePair is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
Definition at line 371 of file reduce_by_key.hpp.
auto ReducePair  (  const ReduceFunction &  reduce_function, 
const ReduceConfig &  reduce_config,  
const KeyHashFunction &  key_hash_function  
)  const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type. The reduce_function is allowed to change the key. Since ReducePair is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
Definition at line 383 of file reduce_by_key.hpp.
auto ReducePair  (  const ReduceFunction &  reduce_function, 
const ReduceConfig &  reduce_config,  
const KeyHashFunction &  key_hash_function,  
const KeyEqualFunction &  key_equal_function  
)  const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type. The reduce_function is allowed to change the key. Since ReducePair is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
key_equal_function  Function to compare keys in reduce hash tables. 
Definition at line 396 of file reduce_by_key.hpp.
References thrill::api::NoDuplicateDetectionTag.
auto ReducePair  (  const DuplicateDetectionFlag< DuplicateDetectionValue > &  , 
const ReduceFunction &  reduce_function,  
const ReduceConfig &  reduce_config = ReduceConfig() , 

const KeyHashFunction &  key_hash_function = KeyHashFunction() , 

const KeyEqualFunction &  key_equal_function = KeyEqualFunction() 

)  const 
ReducePair is a DOp, which groups keyvaluepairs in the input DIA by their key and reduces each keybucket to a single element using the associative reduce_function.
The reduce_function defines how two elements can be reduced to a single element of equal type. The reduce_function is allowed to change the key. Since ReducePair is a DOp, it creates a new DIANode. The DIA returned by Reduce links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of Reduce, as a reduced element can directly be chained to the following LOps.
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
reduce_config  Reduce configuration. 
key_hash_function  Function to hash keys extracted by KeyExtractor. 
key_equal_function  Function to compare keys in reduce hash tables. 
Definition at line 411 of file reduce_by_key.hpp.
References gen_data::value.
auto ReduceToIndex  (  const KeyExtractor &  key_extractor, 
const ReduceFunction &  reduce_function,  
size_t  size,  
const ValueType &  neutral_element = ValueType() , 

const ReduceConfig &  reduce_config = ReduceConfig() 

)  const 
ReduceToIndex is a DOp, which groups elements of the DIA with the key_extractor returning an unsigned integers and reduces each keybucket to a single element using the associative reduce_function.
In contrast to ReduceBy, ReduceToIndex returns a DIA in a defined order, which has the reduced element with key i in position i.
The reduce_function defines how two elements can be reduced to a single element of equal type. The key of the reduced element has to be equal to the keys of the input elements. Since ReduceToIndex is a DOp, it creates a new DIANode. The DIA returned by ReduceToIndex links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of ReduceToIndex, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
size  Resulting DIA size. Consequently, the key_extractor function but always return < size for any element in the input DIA. 
neutral_element  Item value with which to start the reduction in each array cell. 
reduce_config  Reduce configuration. 
Definition at line 260 of file reduce_to_index.hpp.
References thrill::api::NoVolatileKeyTag.
auto ReduceToIndex  (  const VolatileKeyFlag< VolatileKeyValue > &  , 
const KeyExtractor &  key_extractor,  
const ReduceFunction &  reduce_function,  
size_t  size,  
const ValueType &  neutral_element = ValueType() , 

const ReduceConfig &  reduce_config = ReduceConfig() 

)  const 
ReduceToIndex is a DOp, which groups elements of the DIA with the key_extractor returning an unsigned integers and reduces each keybucket to a single element using the associative reduce_function.
In contrast to ReduceByKey, ReduceToIndex returns a DIA in a defined order, which has the reduced element with key i in position i. The reduce_function defines how two elements can be reduced to a single element of equal type.
ReduceToIndex is the equivalent to ReduceByKey, as the reduce_function is allowed to change the key. Since ReduceToIndex is a DOp, it creates a new DIANode. The DIA returned by ReduceToIndex links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of ReduceToIndex, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
size  Resulting DIA size. Consequently, the key_extractor function but always return < size for any element in the input DIA. 
neutral_element  Item value with which to start the reduction in each array cell. 
reduce_config  Reduce configuration. 
Definition at line 275 of file reduce_to_index.hpp.
References gen_data::value.
auto ReduceToIndex  (  const struct SkipPreReducePhaseTag &  , 
const KeyExtractor &  key_extractor,  
const ReduceFunction &  reduce_function,  
size_t  size,  
const ValueType &  neutral_element = ValueType() , 

const ReduceConfig &  reduce_config = ReduceConfig() 

)  const 
ReduceToIndex is a DOp, which groups elements of the DIA with the key_extractor returning an unsigned integers and reduces each keybucket to a single element using the associative reduce_function.
In contrast to ReduceByKey, ReduceToIndex returns a DIA in a defined order, which has the reduced element with key i in position i. The reduce_function defines how two elements can be reduced to a single element of equal type.
ReduceToIndex is the equivalent to ReduceByKey, as the reduce_function is allowed to change the key. Since ReduceToIndex is a DOp, it creates a new DIANode. The DIA returned by ReduceToIndex links to this newly created DIANode. The stack_ of the returned DIA consists of the PostOp of ReduceToIndex, as a reduced element can directly be chained to the following LOps.
key_extractor  Key extractor function, which maps each element to a key of possibly different type. 
ReduceFunction  Type of the reduce_function. This is a function reducing two elements of L's result type to a single element of equal type. 
reduce_function  Reduce function, which defines how the key buckets are reduced to a single element. This function is applied associative but not necessarily commutative. 
size  Resulting DIA size. Consequently, the key_extractor function but always return < size for any element in the input DIA. 
neutral_element  Item value with which to start the reduction in each array cell. 
reduce_config  Reduce configuration. 
Definition at line 333 of file reduce_to_index.hpp.
References gen_data::value.
auto Sort  (  const CompareFunction &  compare_function = CompareFunction()  )  const 
Sort is a DOp, which sorts a given DIA according to the given compare_function.
CompareFunction  Type of the compare_function. Should be (ValueType,ValueType)>bool 
compare_function  Function, which compares two elements. Returns true, if first element is smaller than second. False otherwise. 
Definition at line 740 of file sort.hpp.
References gen_data::value.
auto Sort  (  const CompareFunction &  compare_function, 
const SortFunction &  sort_algorithm  
)  const 
Sort is a DOp, which sorts a given DIA according to the given compare_function.
CompareFunction  Type of the compare_function. Should be (ValueType,ValueType)>bool 
compare_function  Function, which compares two elements. Returns true, if first element is smaller than second. False otherwise. 
sort_algorithm  Algorithm class used to sort items. Merging is always done using a tournament tree with compare_function. 
auto Window  (  size_t  window_size, 
const WindowFunction &  window_function = WindowFunction() 

)  const 
Window is a DOp, which applies a window function to every k consecutive items in a DIA.
The window function is also given the index of the first item, and can output zero or more items via an Emitter.
window_size  the size of the delivered window. Signature: TODO(tb). 
window_function  Window function applied to each k item. 
Definition at line 287 of file window.hpp.
References gen_data::value.
auto Window  (  size_t  window_size, 
const WindowFunction &  window_function,  
const PartialWindowFunction &  partial_window_function  
)  const 
Window is a DOp, which applies a window function to every k consecutive items in a DIA.
The window function is also given the index of the first item, and can output zero or more items via an Emitter.
window_size  the size of the delivered window. Signature: TODO(tb). 
window_function  Window function applied to each k item. 
partial_window_function  Window function applied to less than k items. 
Definition at line 334 of file window.hpp.
References gen_data::value.
auto Window  (  struct DisjointTag const &  , 
size_t  window_size,  
const WindowFunction &  window_function  
)  const 
Window is a DOp, which applies a window function to every k consecutive items in a DIA.
The window function is also given the index of the first item, and can output zero or more items via an Emitter.
window_size  the size of the delivered window. 
window_function  Window function applied to each k item. 
Definition at line 527 of file window.hpp.
References gen_data::value.
auto thrill::api::Zip  (  const ZipFunction &  zip_function, 
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
The two input DIAs are required to be of equal size, otherwise use the CutTag variant.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
first_dia  the initial DIA. 
dias  DIAs, which is zipped together with the original DIA. 
Definition at line 423 of file zip.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), gen_data::value, and tlx::vexpand().
Referenced by BFSNextLevel(), examples::suffix_sorting::CheckSA(), examples::suffix_sorting::ConstructBWT(), and DIA< ValueType_, Stack_ >::Zip().
auto thrill::api::Zip  (  struct CutTag  , 
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips any number of DIAs of equal size in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
If the two input DIAs are of unequal size, the result is the shorter of both. Otherwise use the PadTag variant.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
first_dia  the initial DIA. 
dias  DIAs, which is zipped together with the original DIA. 
Definition at line 476 of file zip.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), gen_data::value, and tlx::vexpand().
auto thrill::api::Zip  (  struct PadTag  , 
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips any number of DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with defaultconstructed items.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
first_dia  the initial DIA. 
dias  DIAs, which is zipped together with the first DIA. 
Definition at line 530 of file zip.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), gen_data::value, and tlx::vexpand().
auto thrill::api::Zip  (  struct PadTag  , 
const ZipFunction &  zip_function,  
const typename common::FunctionTraits< ZipFunction >::args_tuple_plain &  padding,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips any number of DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with items given by the padding parameter.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
padding  std::tuple<args> of padding sentinels delivered to ZipFunction if an input dia is too short. 
first_dia  the initial DIA. 
dias  DIAs, which is zipped together with the original DIA. 
Definition at line 587 of file zip.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), thrill::mem::padding, gen_data::value, and tlx::vexpand().
auto thrill::api::Zip  (  struct NoRebalanceTag  , 
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips any number of DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
In this variant, the DIA partitions on all PEs must have matching length. No rebalancing is performed, and the program will die if any partition mismatches. This enables Zip to proceed without any communication.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
first_dia  the initial DIA. 
dias  DIAs, which is zipped together with the original DIA. 
Definition at line 641 of file zip.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), gen_data::value, and tlx::vexpand().
auto Zip  (  const SecondDIA< ValueType_, Stack_ > &  second_dia, 
const ZipFunction &  zip_function  
)  const 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
The two input DIAs are required to be of equal size, otherwise use the CutTag variant.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
second_dia  DIA, which is zipped together with the original DIA. 
Definition at line 675 of file zip.hpp.
References thrill::api::Zip().
auto Zip  (  struct CutTag const &  , 
const SecondDIA< ValueType_, Stack_ > &  second_dia,  
const ZipFunction &  zip_function  
)  const 
Zips two DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
If the two input DIAs are of unequal size, the result is the shorter of both. Otherwise use PadTag().
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
second_dia  DIA, which is zipped together with the original DIA. 
Definition at line 682 of file zip.hpp.
References thrill::api::Zip().
auto Zip  (  struct PadTag const &  , 
const SecondDIA< ValueType_, Stack_ > &  second_dia,  
const ZipFunction &  zip_function  
)  const 
Zips two DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with defaultconstructed items.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
second_dia  DIA, which is zipped together with the original DIA. 
Definition at line 690 of file zip.hpp.
References thrill::api::Zip().
auto Zip  (  struct NoRebalanceTag const &  , 
const SecondDIA< ValueType_, Stack_ > &  second_dia,  
const ZipFunction &  zip_function  
)  const 
Zips two DIAs in style of functional programming by applying zip_function to the ith elements of both input DIAs to form the ith element of the output DIA.
The type of the output DIA can be inferred from the zip_function.
In this variant, the DIA partitions on all PEs must have matching length. No rebalancing is performed, and the program will die if any partition mismatches. This enables Zip to proceed without any communication.
ZipFunction  Type of the zip_function. This is a function with two input elements, both of the local type, and one output element, which is the type of the Zip node. 
zip_function  Zip function, which zips two elements together 
second_dia  DIA, which is zipped together with the original DIA. 
Definition at line 698 of file zip.hpp.
References thrill::api::Zip().
auto thrill::api::ZipWindow  (  const std::array< size_t, 1+sizeof...(DIAs)> &  window_size, 
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA.
The input DIAs length must be multiples of the corresponding window size. The type of the output DIA can be inferred from the zip_function.
The two input DIAs are required to be of equal window multiples, otherwise use the CutTag variant.
Definition at line 622 of file zip_window.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), gen_data::value, and tlx::vexpand().
Referenced by examples::suffix_sorting::DC3Recursive(), examples::suffix_sorting::DC7Recursive(), and thrill::api::ZipWindow().
auto thrill::api::ZipWindow  (  struct CutTag  , 
const std::array< size_t, 1+sizeof...(DIAs)> &  window_size,  
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA.
The input DIAs length must be multiples of the corresponding window size. The type of the output DIA can be inferred from the zip_function.
If the two input DIAs are of unequal size, the result is the shorter of both. Otherwise use the PadTag variant.
Definition at line 668 of file zip_window.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), gen_data::value, and tlx::vexpand().
auto thrill::api::ZipWindow  (  struct PadTag  , 
const std::array< size_t, 1+sizeof...(DIAs)> &  window_size,  
const ZipFunction &  zip_function,  
const typename ZipWindowTraits< ZipFunction >::value_type_tuple_plain &  padding,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA.
The input DIAs length must be multiples of the corresponding window size. The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with items given by the padding parameter.
Definition at line 715 of file zip_window.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), thrill::mem::padding, gen_data::value, and tlx::vexpand().
auto thrill::api::ZipWindow  (  struct PadTag  , 
const std::array< size_t, 1+sizeof...(DIAs)> &  window_size,  
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA.
The input DIAs length must be multiples of the corresponding window size. The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with defaultconstructed items.
Definition at line 761 of file zip_window.hpp.
References thrill::api::ZipWindow().
auto thrill::api::ZipWindow  (  struct ArrayTag  , 
struct PadTag  ,  
const std::array< size_t, 1+sizeof...(DIAs)> &  window_size,  
const ZipFunction &  zip_function,  
const typename ZipWindowTraits< ZipFunction >::value_type_tuple_plain &  padding,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA.
The input DIAs length must be multiples of the corresponding window size. The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with items given by the padding parameter.
Definition at line 790 of file zip_window.hpp.
References DIA< ValueType_, Stack_ >::AssertValid(), thrill::mem::padding, and tlx::vexpand().
auto thrill::api::ZipWindow  (  struct ArrayTag  , 
struct PadTag  ,  
const std::array< size_t, 1+sizeof...(DIAs)> &  window_size,  
const ZipFunction &  zip_function,  
const DIA< FirstDIAType, FirstDIAStack > &  first_dia,  
const DIAs &...  dias  
) 
Zips two DIAs of equal size in style of functional programming by applying zip_function to the ith fixedsized windows of both input DIAs to form the ith element of the output DIA.
The input DIAs length must be multiples of the corresponding window size. The type of the output DIA can be inferred from the zip_function.
The output DIA's length is the maximum of all input DIAs, shorter DIAs are padded with default constructed items.
Definition at line 837 of file zip_window.hpp.
References thrill::api::ZipWindow().
auto ZipWithIndex  (  const ZipFunction &  zip_function  )  const 
Zips each item of a DIA with its zerobased array index.
This requires a full data store/retrieve cycle because the input DIA's size is generally unknown.
zip_function  Zip function, which gets each element together with its array index. 
Definition at line 141 of file zip_with_index.hpp.
References gen_data::value.