Thrill  0.1

## Actions

size_t Size () const
Computes the total size of all elements across all workers. More...

Future< size_t > SizeFuture () const
Lazily computes the total size of all elements across all workers. More...

std::vector< ValueTypeAllGather () const
Returns the whole DIA in an std::vector on each worker. More...

void AllGather (std::vector< ValueType > *out_vector) const
AllGather is an Action, which returns the whole DIA in an std::vector on each worker. This is only for testing purposes and should not be used on large datasets. More...

Future< std::vector< ValueType > > AllGatherFuture () const
Returns the whole DIA in an std::vector on each worker. More...

void Print (const std::string &name=std::string()) const
Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization. More...

void Print (const std::string &name, std::ostream &out) const
Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization. More...

std::vector< ValueTypeGather (size_t target_id=0) const
Gather is an Action, which collects all data of the DIA into a vector at the given worker. More...

void Gather (size_t target_id, std::vector< ValueType > *out_vector) const
Gather is an Action, which collects all data of the DIA into a vector at the given worker. More...

template<typename ReduceFunction >
ValueType AllReduce (const ReduceFunction &reduce_function) const
AllReduce is an Action, which computes the reduction sum of all elements globally and delivers the same value on all workers. More...

template<typename ReduceFunction >
ValueType AllReduce (const ReduceFunction &reduce_function, const ValueType &initial_value) const
AllReduce is an Action, which computes the reduction sum of all elements globally and delivers the same value on all workers. More...

template<typename ReduceFunction >
Future< ValueTypeAllReduceFuture (const ReduceFunction &reduce_function) const
AllReduce is an ActionFuture, which computes the reduction sum of all elements globally and delivers the same value on all workers. More...

template<typename ReduceFunction >
Future< ValueTypeAllReduceFuture (const ReduceFunction &reduce_function, const ValueType &initial_value) const
AllReduce is an ActionFuture, which computes the reduction sum of all elements globally and delivers the same value on all workers. More...

template<typename SumFunction = std::plus<ValueType>>
ValueType Sum (const SumFunction &sum_function=SumFunction()) const
Sum is an Action, which computes the sum of all elements globally. More...

template<typename SumFunction = std::plus<ValueType>>
ValueType Sum (const SumFunction &sum_function, const ValueType &initial_value) const
Sum is an Action, which computes the sum of all elements globally. More...

template<typename SumFunction = std::plus<ValueType>>
Future< ValueTypeSumFuture (const SumFunction &sum_function=SumFunction(), const ValueType &initial_value=ValueType()) const
Sum is an ActionFuture, which computes the sum of all elements globally. More...

ValueType Min () const
Min is an Action, which computes the minimum of all elements globally. More...

ValueType Min (const ValueType &initial_value) const
Min is an Action, which computes the minimum of all elements globally. More...

Future< ValueTypeMinFuture () const
Min is an ActionFuture, which computes the minimum of all elements globally. More...

Future< ValueTypeMinFuture (const ValueType &initial_value) const
Min is an ActionFuture, which computes the minimum of all elements globally. More...

ValueType Max () const
Max is an Action, which computes the maximum of all elements globally. More...

ValueType Max (const ValueType &initial_value) const
Max is an Action, which computes the maximum of all elements globally. More...

Future< ValueTypeMaxFuture () const
Max is an ActionFuture, which computes the maximum of all elements globally. More...

Future< ValueTypeMaxFuture (const ValueType &initial_value) const
Max is an ActionFuture, which computes the maximum of all elements globally. More...

template<size_t p>
double HyperLogLog () const
Compute the approximate number of distinct elements in the DIA. More...

void WriteLinesOne (const std::string &filepath) const
WriteLinesOne is an Action, which writes std::strings to a single output file. More...

Future< void > WriteLinesOneFuture (const std::string &filepath) const
WriteLinesOne is an ActionFuture, which writes std::strings to a single output file. More...

void WriteLines (const std::string &filepath, size_t target_file_size=128 *1024 *1024) const
WriteLines is an Action, which writes std::strings to multiple output files. More...

Future< void > WriteLinesFuture (const std::string &filepath, size_t target_file_size=128 *1024 *1024) const
WriteLines is an ActionFuture, which writes std::strings to multiple output files. More...

void WriteBinary (const std::string &filepath, size_t max_file_size=128 *1024 *1024) const
WriteBinary is a function, which writes a DIA to many files per worker. More...

Future< void > WriteBinaryFuture (const std::string &filepath, size_t max_file_size=128 *1024 *1024) const
WriteBinary is a function, which writes a DIA to many files per worker. More...

auto Sample (size_t sample_size) const
Select up to sample_size items uniformly at random and return a new DIA<T>. More...

## ◆ AllGather() [1/2]

 std::vector< ValueType > AllGather ( ) const

Returns the whole DIA in an std::vector on each worker.

This is only for testing purposes and should not be used on large datasets.

Definition at line 114 of file all_gather.hpp.

Referenced by Process().

## ◆ AllGather() [2/2]

 void AllGather ( std::vector< ValueType > * out_vector ) const

AllGather is an Action, which returns the whole DIA in an std::vector on each worker. This is only for testing purposes and should not be used on large datasets.

Definition at line 130 of file all_gather.hpp.

## ◆ AllGatherFuture()

 Future< std::vector< ValueType > > AllGatherFuture ( ) const

Returns the whole DIA in an std::vector on each worker.

This is only for testing purposes and should not be used on large datasets.

Definition at line 143 of file all_gather.hpp.

## ◆ AllReduce() [1/2]

 ValueType AllReduce ( const ReduceFunction & reduce_function ) const

AllReduce is an Action, which computes the reduction sum of all elements globally and delivers the same value on all workers.

Parameters
 reduce_function Reduce function.

Definition at line 89 of file all_reduce.hpp.

## ◆ AllReduce() [2/2]

 ValueType AllReduce ( const ReduceFunction & reduce_function, const ValueType & initial_value ) const

AllReduce is an Action, which computes the reduction sum of all elements globally and delivers the same value on all workers.

Parameters
 reduce_function Reduce function. initial_value Initial value of the reduction.

Definition at line 124 of file all_reduce.hpp.

## ◆ AllReduceFuture() [1/2]

 Future< ValueType > AllReduceFuture ( const ReduceFunction & reduce_function ) const

AllReduce is an ActionFuture, which computes the reduction sum of all elements globally and delivers the same value on all workers.

Parameters
 reduce_function Reduce function.

Definition at line 159 of file all_reduce.hpp.

## ◆ AllReduceFuture() [2/2]

 Future< ValueType > AllReduceFuture ( const ReduceFunction & reduce_function, const ValueType & initial_value ) const

AllReduce is an ActionFuture, which computes the reduction sum of all elements globally and delivers the same value on all workers.

Parameters
 reduce_function Reduce function. initial_value Initial value of the reduction.

Definition at line 192 of file all_reduce.hpp.

## ◆ Gather() [1/2]

 std::vector< ValueType > Gather ( size_t target_id = 0 ) const

Gather is an Action, which collects all data of the DIA into a vector at the given worker.

This should only be done if the received data can fit into RAM of the one worker.

Definition at line 93 of file gather.hpp.

Referenced by examples::select::Select().

## ◆ Gather() [2/2]

 void Gather ( size_t target_id, std::vector< ValueType > * out_vector ) const

Gather is an Action, which collects all data of the DIA into a vector at the given worker.

This should only be done if the received data can fit into RAM of the one worker.

Definition at line 109 of file gather.hpp.

## ◆ HyperLogLog()

 double HyperLogLog ( ) const

Compute the approximate number of distinct elements in the DIA.

Template Parameters
 p Number of bits to use for index. Should be between 4 and 16.

Definition at line 64 of file hyperloglog.hpp.

## ◆ Max() [1/2]

 ValueType Max ( ) const

Max is an Action, which computes the maximum of all elements globally.

Definition at line 22 of file max.hpp.

## ◆ Max() [2/2]

 ValueType Max ( const ValueType & initial_value ) const

Max is an Action, which computes the maximum of all elements globally.

Parameters
 initial_value Initial value of the max.

Definition at line 32 of file max.hpp.

## ◆ MaxFuture() [1/2]

 Future< ValueType > MaxFuture ( ) const

Max is an ActionFuture, which computes the maximum of all elements globally.

Definition at line 43 of file max.hpp.

## ◆ MaxFuture() [2/2]

 Future< ValueType > MaxFuture ( const ValueType & initial_value ) const

Max is an ActionFuture, which computes the maximum of all elements globally.

Parameters
 initial_value Initial value of the max.

Definition at line 52 of file max.hpp.

## ◆ Min() [1/2]

 ValueType Min ( ) const

Min is an Action, which computes the minimum of all elements globally.

Definition at line 22 of file min.hpp.

## ◆ Min() [2/2]

 ValueType Min ( const ValueType & initial_value ) const

Min is an Action, which computes the minimum of all elements globally.

Parameters
 initial_value Initial value of the min.

Definition at line 32 of file min.hpp.

## ◆ MinFuture() [1/2]

 Future< ValueType > MinFuture ( ) const

Min is an ActionFuture, which computes the minimum of all elements globally.

Definition at line 43 of file min.hpp.

## ◆ MinFuture() [2/2]

 Future< ValueType > MinFuture ( const ValueType & initial_value ) const

Min is an ActionFuture, which computes the minimum of all elements globally.

Parameters
 initial_value Initial value of the min.

Definition at line 52 of file min.hpp.

## ◆ Print() [1/2]

 void Print ( const std::string & name = std::string() ) const

Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization.

It is implemented using Gather().

Definition at line 50 of file print.hpp.

## ◆ Print() [2/2]

 void Print ( const std::string & name, std::ostream & out ) const

Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization.

It is implemented using Gather().

Definition at line 24 of file print.hpp.

## ◆ Sample()

 auto Sample ( size_t sample_size ) const

Select up to sample_size items uniformly at random and return a new DIA<T>.

Examples:
examples/tutorial/k-means_step4.cpp.

Definition at line 247 of file sample.hpp.

Referenced by Process().

## ◆ Size()

 size_t Size ( ) const

Computes the total size of all elements across all workers.

Definition at line 87 of file size.hpp.

Referenced by examples::select::Select().

## ◆ SizeFuture()

 Future< size_t > SizeFuture ( ) const

Lazily computes the total size of all elements across all workers.

Definition at line 97 of file size.hpp.

## ◆ Sum() [1/2]

 ValueType Sum ( const SumFunction & sum_function = SumFunction() ) const

Sum is an Action, which computes the sum of all elements globally.

Parameters
 sum_function Sum function.

Definition at line 23 of file sum.hpp.

Referenced by PrepareNextTree().

## ◆ Sum() [2/2]

 ValueType Sum ( const SumFunction & sum_function, const ValueType & initial_value ) const

Sum is an Action, which computes the sum of all elements globally.

Parameters
 sum_function Sum function. initial_value Initial value of the sum.

Definition at line 57 of file sum.hpp.

## ◆ SumFuture()

 Future< ValueType > SumFuture ( const SumFunction & sum_function = SumFunction(), const ValueType & initial_value = ValueType() ) const

Sum is an ActionFuture, which computes the sum of all elements globally.

Parameters
 sum_function Sum function. initial_value Initial value of the sum.

Definition at line 92 of file sum.hpp.

## ◆ WriteBinary()

 void WriteBinary ( const std::string & filepath, size_t max_file_size = 128* 1024* 1024 ) const

WriteBinary is a function, which writes a DIA to many files per worker.

The input DIA can be recreated with ReadBinary and equal filepath.

Parameters
 filepath Destination of the output file. This filepath must contain two special substrings: "$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "" and/or "##########" are automatically appended. max_file_size size limit of individual file.

Definition at line 191 of file write_binary.hpp.

## ◆ WriteBinaryFuture()

 Future< void > WriteBinaryFuture ( const std::string & filepath, size_t max_file_size = 128* 1024* 1024 ) const

WriteBinary is a function, which writes a DIA to many files per worker.

The input DIA can be recreated with ReadBinary and equal filepath.

Parameters
 filepath Destination of the output file. This filepath must contain two special substrings: "$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "" and/or "##########" are automatically appended. max_file_size size limit of individual file.

Definition at line 203 of file write_binary.hpp.

## ◆ WriteLines()

 void WriteLines ( const std::string & filepath, size_t target_file_size = 128* 1024* 1024 ) const

WriteLines is an Action, which writes std::strings to multiple output files.

Strings are written using fstream with a newline after each entry. Each worker creates its individual file.

Parameters
 filepath Destination of the output file. This filepath must contain two special substrings: "$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "" and/or "##########" are automatically appended. target_file_size target size of each individual file.

Definition at line 176 of file write_lines.hpp.

## ◆ WriteLinesFuture()

 Future< void > WriteLinesFuture ( const std::string & filepath, size_t target_file_size = 128* 1024* 1024 ) const

WriteLines is an ActionFuture, which writes std::strings to multiple output files.

Strings are written using fstream with a newline after each entry. Each worker creates its individual file.

Parameters
 filepath Destination of the output file. This filepath must contain two special substrings: "$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "" and/or "##########" are automatically appended. target_file_size target size of each individual file.

Definition at line 192 of file write_lines.hpp.

## ◆ WriteLinesOne()

 void WriteLinesOne ( const std::string & filepath ) const

WriteLinesOne is an Action, which writes std::strings to a single output file.

Parameters
 filepath Destination of the output file.

Definition at line 111 of file write_lines_one.hpp.

## ◆ WriteLinesOneFuture()

 Future< void > WriteLinesOneFuture ( const std::string & filepath ) const

WriteLinesOne is an ActionFuture, which writes std::strings to a single output file.

Parameters
 filepath Destination of the output file.

Definition at line 126 of file write_lines_one.hpp.