Thrill  0.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
+ Collaboration diagram for Actions:

Actions

size_t Size () const
 Computes the total size of all elements across all workers. More...
 
Future< size_t > SizeFuture () const
 Lazily computes the total size of all elements across all workers. More...
 
std::vector< ValueType > AllGather () const
 Returns the whole DIA in an std::vector on each worker. More...
 
void AllGather (std::vector< ValueType > *out_vector) const
 AllGather is an Action, which returns the whole DIA in an std::vector on each worker. This is only for testing purposes and should not be used on large datasets. More...
 
Future< std::vector< ValueType > > AllGatherFuture () const
 Returns the whole DIA in an std::vector on each worker. More...
 
void Print (const std::string &name) const
 Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization. More...
 
void Print (const std::string &name, std::ostream &out) const
 Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization. More...
 
std::vector< ValueType > Gather (size_t target_id=0) const
 Gather is an Action, which collects all data of the DIA into a vector at the given worker. More...
 
void Gather (size_t target_id, std::vector< ValueType > *out_vector) const
 Gather is an Action, which collects all data of the DIA into a vector at the given worker. More...
 
template<typename ReduceFunction >
ValueType AllReduce (const ReduceFunction &reduce_function, const ValueType &initial_value=ValueType()) const
 AllReduce is an Action, which computes the reduction sum of all elements globally and delivers the same value on all workers. More...
 
template<typename ReduceFunction >
Future< ValueType > AllReduceFuture (const ReduceFunction &reduce_function, const ValueType &initial_value=ValueType()) const
 AllReduce is an ActionFuture, which computes the reduction sum of all elements globally and delivers the same value on all workers. More...
 
template<typename SumFunction = std::plus<ValueType>>
ValueType Sum (const SumFunction &sum_function=SumFunction(), const ValueType &initial_value=ValueType()) const
 Sum is an Action, which computes the sum of all elements globally. More...
 
template<typename SumFunction = std::plus<ValueType>>
Future< ValueType > SumFuture (const SumFunction &sum_function=SumFunction(), const ValueType &initial_value=ValueType()) const
 Sum is an ActionFuture, which computes the sum of all elements globally. More...
 
ValueType Min (const ValueType &initial_value=ValueType()) const
 Min is an Action, which computes the minimum of all elements globally. More...
 
Future< ValueType > MinFuture (const ValueType &initial_value=ValueType()) const
 Min is an ActionFuture, which computes the minimum of all elements globally. More...
 
ValueType Max (const ValueType &initial_value=ValueType()) const
 Max is an Action, which computes the maximum of all elements globally. More...
 
Future< ValueType > MaxFuture (const ValueType &initial_value=ValueType()) const
 Max is an ActionFuture, which computes the maximum of all elements globally. More...
 
template<size_t p>
double HyperLogLog () const
 Compute the approximate number of distinct elements in the DIA. More...
 
void WriteLinesOne (const std::string &filepath) const
 WriteLinesOne is an Action, which writes std::strings to a single output file. More...
 
Future< void > WriteLinesOneFuture (const std::string &filepath) const
 WriteLinesOne is an ActionFuture, which writes std::strings to a single output file. More...
 
void WriteLines (const std::string &filepath, size_t target_file_size=128 *1024 *1024) const
 WriteLines is an Action, which writes std::strings to multiple output files. More...
 
Future< void > WriteLinesFuture (const std::string &filepath, size_t target_file_size=128 *1024 *1024) const
 WriteLines is an ActionFuture, which writes std::strings to multiple output files. More...
 
void WriteBinary (const std::string &filepath, size_t max_file_size=128 *1024 *1024) const
 WriteBinary is a function, which writes a DIA to many files per worker. More...
 
Future< void > WriteBinaryFuture (const std::string &filepath, size_t max_file_size=128 *1024 *1024) const
 WriteBinary is a function, which writes a DIA to many files per worker. More...
 
auto Sample (size_t sample_size) const
 Select up to sample_size items uniformly at random and return a new DIA<T>. More...
 

Detailed Description

Function Documentation

std::vector< ValueType > AllGather ( ) const

Returns the whole DIA in an std::vector on each worker.

This is only for testing purposes and should not be used on large datasets.

Definition at line 115 of file all_gather.hpp.

References DIABase::RunScope().

void AllGather ( std::vector< ValueType > *  out_vector) const

AllGather is an Action, which returns the whole DIA in an std::vector on each worker. This is only for testing purposes and should not be used on large datasets.

Definition at line 131 of file all_gather.hpp.

References DIABase::RunScope().

Future< std::vector< ValueType > > AllGatherFuture ( ) const

Returns the whole DIA in an std::vector on each worker.

This is only for testing purposes and should not be used on large datasets.

Definition at line 144 of file all_gather.hpp.

ValueType AllReduce ( const ReduceFunction &  reduce_function,
const ValueType initial_value = ValueType() 
) const

AllReduce is an Action, which computes the reduction sum of all elements globally and delivers the same value on all workers.

Parameters
reduce_functionReduce function.
initial_valueInitial value of the reduction.

Definition at line 88 of file all_reduce.hpp.

References gen_data::value.

Future< ValueType > AllReduceFuture ( const ReduceFunction &  reduce_function,
const ValueType initial_value = ValueType() 
) const

AllReduce is an ActionFuture, which computes the reduction sum of all elements globally and delivers the same value on all workers.

Parameters
reduce_functionReduce function.
initial_valueInitial value of the reduction.

Definition at line 122 of file all_reduce.hpp.

References gen_data::value.

std::vector< ValueType > Gather ( size_t  target_id = 0) const

Gather is an Action, which collects all data of the DIA into a vector at the given worker.

This should only be done if the received data can fit into RAM of the one worker.

Definition at line 94 of file gather.hpp.

References DIABase::RunScope().

void Gather ( size_t  target_id,
std::vector< ValueType > *  out_vector 
) const

Gather is an Action, which collects all data of the DIA into a vector at the given worker.

This should only be done if the received data can fit into RAM of the one worker.

Definition at line 110 of file gather.hpp.

References DIABase::RunScope().

double HyperLogLog ( ) const

Compute the approximate number of distinct elements in the DIA.

Template Parameters
pNumber of bits to use for index. Should be between 4 and 16.

Definition at line 64 of file hyperloglog.hpp.

ValueType Max ( const ValueType initial_value = ValueType()) const

Max is an Action, which computes the maximum of all elements globally.

Parameters
initial_valueInitial value of the max.

Definition at line 22 of file max.hpp.

References DIABase::RunScope().

Future< ValueType > MaxFuture ( const ValueType initial_value = ValueType()) const

Max is an ActionFuture, which computes the maximum of all elements globally.

Parameters
initial_valueInitial value of the max.

Definition at line 32 of file max.hpp.

ValueType Min ( const ValueType initial_value = ValueType()) const

Min is an Action, which computes the minimum of all elements globally.

Parameters
initial_valueInitial value of the min.

Definition at line 22 of file min.hpp.

References DIABase::RunScope().

Future< ValueType > MinFuture ( const ValueType initial_value = ValueType()) const

Min is an ActionFuture, which computes the minimum of all elements globally.

Parameters
initial_valueInitial value of the min.

Definition at line 32 of file min.hpp.

void Print ( const std::string &  name) const

Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization.

It is implemented using Gather().

Definition at line 48 of file print.hpp.

void Print ( const std::string &  name,
std::ostream &  out 
) const

Print is an Action, which collects all data of the DIA at the worker 0 and prints using ostream serialization.

It is implemented using Gather().

Definition at line 24 of file print.hpp.

References DIABase::RunScope().

auto Sample ( size_t  sample_size) const

Select up to sample_size items uniformly at random and return a new DIA<T>.

Definition at line 247 of file sample.hpp.

size_t Size ( ) const

Computes the total size of all elements across all workers.

Definition at line 100 of file size.hpp.

References DIABase::RunScope().

Future< size_t > SizeFuture ( ) const

Lazily computes the total size of all elements across all workers.

Definition at line 110 of file size.hpp.

ValueType Sum ( const SumFunction &  sum_function = SumFunction(),
const ValueType initial_value = ValueType() 
) const

Sum is an Action, which computes the sum of all elements globally.

Parameters
sum_functionSum function.
initial_valueInitial value of the sum.

Definition at line 23 of file sum.hpp.

References gen_data::value.

Future< ValueType > SumFuture ( const SumFunction &  sum_function = SumFunction(),
const ValueType initial_value = ValueType() 
) const

Sum is an ActionFuture, which computes the sum of all elements globally.

Parameters
sum_functionSum function.
initial_valueInitial value of the sum.

Definition at line 57 of file sum.hpp.

References gen_data::value.

void WriteBinary ( const std::string &  filepath,
size_t  max_file_size = 128* 1024* 1024 
) const

WriteBinary is a function, which writes a DIA to many files per worker.

The input DIA can be recreated with ReadBinary and equal filepath.

Parameters
filepathDestination of the output file. This filepath must contain two special substrings: "$$$$$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "$$$$" and/or "##########" are automatically appended.
max_file_sizesize limit of individual file.

Definition at line 192 of file write_binary.hpp.

References DIABase::RunScope().

Future< void > WriteBinaryFuture ( const std::string &  filepath,
size_t  max_file_size = 128* 1024* 1024 
) const

WriteBinary is a function, which writes a DIA to many files per worker.

The input DIA can be recreated with ReadBinary and equal filepath.

Parameters
filepathDestination of the output file. This filepath must contain two special substrings: "$$$$$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "$$$$" and/or "##########" are automatically appended.
max_file_sizesize limit of individual file.

Definition at line 204 of file write_binary.hpp.

void WriteLines ( const std::string &  filepath,
size_t  target_file_size = 128* 1024* 1024 
) const

WriteLines is an Action, which writes std::strings to multiple output files.

Strings are written using fstream with a newline after each entry. Each worker creates its individual file.

Parameters
filepathDestination of the output file. This filepath must contain two special substrings: "$$$$$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "$$$$" and/or "##########" are automatically appended.
target_file_sizetarget size of each individual file.

Definition at line 177 of file write_lines.hpp.

References DIABase::RunScope(), and gen_data::value.

Future< void > WriteLinesFuture ( const std::string &  filepath,
size_t  target_file_size = 128* 1024* 1024 
) const

WriteLines is an ActionFuture, which writes std::strings to multiple output files.

Strings are written using fstream with a newline after each entry. Each worker creates its individual file.

Parameters
filepathDestination of the output file. This filepath must contain two special substrings: "$$$$$" is replaced by the worker id and "#####" will be replaced by the file chunk id. The last occurrences of "$" and "#" are replaced, otherwise "$$$$" and/or "##########" are automatically appended.
target_file_sizetarget size of each individual file.

Definition at line 193 of file write_lines.hpp.

References gen_data::value.

void WriteLinesOne ( const std::string &  filepath) const

WriteLinesOne is an Action, which writes std::strings to a single output file.

Parameters
filepathDestination of the output file.

Definition at line 112 of file write_lines_one.hpp.

References DIABase::RunScope(), and gen_data::value.

Future< void > WriteLinesOneFuture ( const std::string &  filepath) const

WriteLinesOne is an ActionFuture, which writes std::strings to a single output file.

Parameters
filepathDestination of the output file.

Definition at line 127 of file write_lines_one.hpp.

References gen_data::value.