Thrill  0.1
Source Nodes
Collaboration diagram for Source Nodes:

## Functions

template<typename ValueType >
auto ConcatToDIA (Context &ctx, const std::vector< ValueType > &in_vector)
ConcatToDIA is a Source-DOp, which takes a vector of data on all workers, and CONCATENATES them into a DIA. More...

template<typename ValueType >
auto EqualToDIA (Context &ctx, const std::vector< ValueType > &in_vector)
EqualToDIA is a Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA. More...

template<typename ValueType >
auto EqualToDIA (Context &ctx, std::vector< ValueType > &&in_vector)
EqualToDIA is an Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA. More...

template<typename GenerateFunction >
auto Generate (Context &ctx, size_t size, const GenerateFunction &generate_function)
Generate is a Source-DOp, which creates a DIA of given size using a generator function. More...

static auto Generate (Context &ctx, size_t size)
Generate is a Source-DOp, which creates a DIA of given size containing the size_t indexes [0,size). More...

template<typename ValueType >
DIA< ValueType > ReadBinary (Context &ctx, const std::vector< std::string > &filepath, uint64_t size_limit=ReadBinaryNode< ValueType >::no_size_limit_)
ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA. More...

template<typename ValueType >
DIA< ValueType > ReadBinary (Context &ctx, const std::string &filepath, uint64_t size_limit=ReadBinaryNode< ValueType >::no_size_limit_)
ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA. More...

DIA< std::string > ReadLines (Context &ctx, const std::string &filepath)
ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function. More...

DIA< std::string > ReadLines (Context &ctx, const std::vector< std::string > &filepaths)
ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function. More...

template<typename ValueType >
auto Distribute (Context &ctx, const std::vector< ValueType > &in_vector, size_t source_id=0)
Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA. More...

template<typename ValueType >
auto Distribute (Context &ctx, std::vector< ValueType > &&in_vector, size_t source_id=0)
Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA. More...

## Function Documentation

 auto thrill::api::ConcatToDIA ( Context & ctx, const std::vector< ValueType > & in_vector )

ConcatToDIA is a Source-DOp, which takes a vector of data on all workers, and CONCATENATES them into a DIA.

Use Distribute to actually distribute data from a single worker, ConcatToDIA is a wrapper if the data is already distributed.

Parameters
 ctx Reference to the Context object in_vector Vector to concatenate into a DIA, the contents is COPIED into the DIANode.

Definition at line 76 of file concat_to_dia.hpp.

 auto thrill::api::Distribute ( Context & ctx, const std::vector< ValueType > & in_vector, size_t source_id = 0 )

Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA.

Definition at line 106 of file distribute.hpp.

 auto thrill::api::Distribute ( Context & ctx, std::vector< ValueType > && in_vector, size_t source_id = 0 )

Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA.

Definition at line 126 of file distribute.hpp.

 auto thrill::api::EqualToDIA ( Context & ctx, const std::vector< ValueType > & in_vector )

EqualToDIA is a Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA.

Use Distribute to actually distribute data from a single worker, EqualToDIA is a wrapper if the data is already distributed.

Parameters
 ctx Reference to the Context object in_vector Vector to convert to a DIA, the contents is COPIED into the DIANode.

Definition at line 79 of file equal_to_dia.hpp.

Referenced by ConstructRLBWT().

 auto thrill::api::EqualToDIA ( Context & ctx, std::vector< ValueType > && in_vector )

EqualToDIA is an Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA.

Use Distribute to actually distribute data from a single worker, EqualToDIA is a wrapper if the data is already distributed.

Parameters
 ctx Reference to the Context object in_vector Vector to convert to a DIA, the contents is MOVED into the DIANode.

Definition at line 101 of file equal_to_dia.hpp.

 auto thrill::api::Generate ( Context & ctx, size_t size, const GenerateFunction & generate_function )

Generate is a Source-DOp, which creates a DIA of given size using a generator function.

The generator function called for each index in the range of [0,size) and must output exactly one item.

Parameters
 ctx Reference to the Context object size Size of the output DIA generate_function Generator function, which maps size_t from [0,size) to elements. Input type has to be size_t.
Examples:
examples/tutorial/k-means_step1.cpp, examples/tutorial/k-means_step2.cpp, examples/tutorial/k-means_step3.cpp, examples/tutorial/k-means_step4.cpp, examples/tutorial/k-means_step5.cpp, and examples/tutorial/k-means_step6.cpp.

Definition at line 85 of file generate.hpp.

References gen_data::value.

 static auto thrill::api::Generate ( Context & ctx, size_t size )
inlinestatic

Generate is a Source-DOp, which creates a DIA of given size containing the size_t indexes [0,size).

Parameters
 ctx Reference to the Context object size Size of the output DIA

Definition at line 122 of file generate.hpp.

References thrill::api::Generate().

 DIA thrill::api::ReadBinary ( Context & ctx, const std::vector< std::string > & filepath, uint64_t size_limit = ReadBinaryNode::no_size_limit_ )

ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA.

Parameters
 ctx Reference to the context object filepath Path of the file in the file system size_limit Optional limit to the total file size (e.g. for testing algorithms on prefixes)

Definition at line 355 of file read_binary.hpp.

 DIA thrill::api::ReadBinary ( Context & ctx, const std::string & filepath, uint64_t size_limit = ReadBinaryNode::no_size_limit_ )

ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA.

Parameters
 ctx Reference to the context object filepath Path of the file in the file system size_limit Optional limit to the total file size (e.g. for testing algorithms on prefixes)

Definition at line 389 of file read_binary.hpp.

 DIA thrill::api::ReadLines ( Context & ctx, const std::string & filepath )

ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function.

Parameters
 ctx Reference to the context object filepath Path of the file in the file system
Examples:
examples/tutorial/k-means_step5.cpp, and examples/tutorial/k-means_step6.cpp.

Definition at line 452 of file read_lines.hpp.

 DIA thrill::api::ReadLines ( Context & ctx, const std::vector< std::string > & filepaths )

ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function.

Parameters
 ctx Reference to the context object filepaths Path of the file in the file system

Definition at line 467 of file read_lines.hpp.