Functions
template<typename ValueType >
auto	ConcatToDIA (Context &ctx, const std::vector< ValueType > &in_vector)
	ConcatToDIA is a Source-DOp, which takes a vector of data on all workers, and CONCATENATES them into a DIA. More...

template<typename ValueType >
auto	EqualToDIA (Context &ctx, const std::vector< ValueType > &in_vector)
	EqualToDIA is a Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA. More...

template<typename ValueType >
auto	EqualToDIA (Context &ctx, std::vector< ValueType > &&in_vector)
	EqualToDIA is an Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA. More...

template<typename GenerateFunction >
auto	Generate (Context &ctx, size_t size, const GenerateFunction &generate_function)
	Generate is a Source-DOp, which creates a DIA of given size using a generator function. More...

static auto	Generate (Context &ctx, size_t size)
	Generate is a Source-DOp, which creates a DIA of given size containing the size_t indexes `[0,size)`. More...

template<typename ValueType >
DIA< ValueType >	ReadBinary (Context &ctx, const std::vector< std::string > &filepath, uint64_t size_limit=ReadBinaryNode< ValueType >::no_size_limit_)
	ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA. More...

template<typename ValueType >
DIA< ValueType >	ReadBinary (struct LocalStorageTag, Context &ctx, const std::vector< std::string > &filepath, uint64_t size_limit=ReadBinaryNode< ValueType >::no_size_limit_)
	ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA. More...

template<typename ValueType >
DIA< ValueType >	ReadBinary (Context &ctx, const std::string &filepath, uint64_t size_limit=ReadBinaryNode< ValueType >::no_size_limit_)
	ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA. More...

template<typename ValueType >
DIA< ValueType >	ReadBinary (struct LocalStorageTag, Context &ctx, const std::string &filepath, uint64_t size_limit=ReadBinaryNode< ValueType >::no_size_limit_)
	ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA. More...

DIA< std::string >	ReadLines (Context &ctx, const std::string &filepath)
	ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function. More...

DIA< std::string >	ReadLines (struct LocalStorageTag, Context &ctx, const std::string &filepath)
	ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function. More...

DIA< std::string >	ReadLines (Context &ctx, const std::vector< std::string > &filepaths)
	ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function. More...

DIA< std::string >	ReadLines (struct LocalStorageTag, Context &ctx, const std::vector< std::string > &filepaths)
	ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function. More...

template<typename ValueType >
auto	Distribute (Context &ctx, const std::vector< ValueType > &in_vector, size_t source_id=0)
	Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA. More...

template<typename ValueType >
auto	Distribute (Context &ctx, std::vector< ValueType > &&in_vector, size_t source_id=0)
	Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA. More...

Detailed Description

Function Documentation

◆ ConcatToDIA()

auto thrill::api::ConcatToDIA	(	Context &	ctx,
		const std::vector< ValueType > &	in_vector
	)

ConcatToDIA is a Source-DOp, which takes a vector of data on all workers, and CONCATENATES them into a DIA.

Use Distribute to actually distribute data from a single worker, ConcatToDIA is a wrapper if the data is already distributed.

Parameters

ctx	Reference to the Context object
in_vector	Vector to concatenate into a DIA, the contents is COPIED into the DIANode.

Definition at line 78 of file concat_to_dia.hpp.

Referenced by thrill::api::ConcatToDIA().

◆ Distribute() [1/2]

auto thrill::api::Distribute	(	Context &	ctx,
		const std::vector< ValueType > &	in_vector,
		size_t	source_id = `0`
	)

Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA.

Definition at line 106 of file distribute.hpp.

Referenced by thrill::api::Distribute().

◆ Distribute() [2/2]

auto thrill::api::Distribute	(	Context &	ctx,
		std::vector< ValueType > &&	in_vector,
		size_t	source_id = `0`
	)

Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA.

Definition at line 126 of file distribute.hpp.

References thrill::api::Distribute().

◆ EqualToDIA() [1/2]

auto thrill::api::EqualToDIA	(	Context &	ctx,
		const std::vector< ValueType > &	in_vector
	)

EqualToDIA is a Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA.

Use Distribute to actually distribute data from a single worker, EqualToDIA is a wrapper if the data is already distributed.

Parameters

ctx	Reference to the Context object
in_vector	Vector to convert to a DIA, the contents is COPIED into the DIANode.

Definition at line 81 of file equal_to_dia.hpp.

Referenced by ConstructRLBWT(), and thrill::api::EqualToDIA().

◆ EqualToDIA() [2/2]

auto thrill::api::EqualToDIA	(	Context &	ctx,
		std::vector< ValueType > &&	in_vector
	)

EqualToDIA is an Source-DOp, which takes a vector of data EQUAL on all workers, and returns the data in a DIA.

Use Distribute to actually distribute data from a single worker, EqualToDIA is a wrapper if the data is already distributed.

Parameters

ctx	Reference to the Context object
in_vector	Vector to convert to a DIA, the contents is MOVED into the DIANode.

Definition at line 105 of file equal_to_dia.hpp.

References thrill::api::EqualToDIA().

◆ Generate() [1/2]

auto thrill::api::Generate	(	Context &	ctx,
		size_t	size,
		const GenerateFunction &	generate_function
	)

Generate is a Source-DOp, which creates a DIA of given size using a generator function.

The generator function called for each index in the range of [0,size) and must output exactly one item.

Parameters

ctx	Reference to the Context object
size	Size of the output DIA
generate_function	Generator function, which maps `size_t` from `[0,size)` to elements. Input type has to be `size_t`.

Examples:: examples/tutorial/k-means_step1.cpp, examples/tutorial/k-means_step2.cpp, examples/tutorial/k-means_step3.cpp, examples/tutorial/k-means_step4.cpp, examples/tutorial/k-means_step5.cpp, and examples/tutorial/k-means_step6.cpp.

Definition at line 87 of file generate.hpp.

References gen_data::value.

Referenced by examples::suffix_sorting::ConstructLCP(), CountTrianglesGenerated(), thrill::api::Generate(), GenerateInput(), GeneratePoints(), GenerateTestData(), main(), examples::page_rank::PageRank(), examples::page_rank::PageRankJoin(), Parallel(), Process(), RunHashWordCountGenerated(), RunKMeansGenerated(), RunPageRankGenerated(), RunPageRankJoinGenerated(), RunSelect(), RunStochasticGradGenerated(), and RunWordCountGenerated().

◆ Generate() [2/2]

static auto thrill::api::Generate	(	Context &	ctx,
		size_t	size
	)

inlinestatic

Generate is a Source-DOp, which creates a DIA of given size containing the size_t indexes [0,size).

Parameters

ctx	Reference to the Context object
size	Size of the output DIA

Definition at line 126 of file generate.hpp.

References thrill::api::Generate().

◆ ReadBinary() [1/4]

DIA<ValueType> thrill::api::ReadBinary	(	Context &	ctx,
		const std::vector< std::string > &	filepath,
		uint64_t	size_limit = `ReadBinaryNode<ValueType>::no_size_limit_`
	)

ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA.

Parameters

ctx	Reference to the context object
filepath	Path of the file in the file system
size_limit	Optional limit to the total file size (e.g. for testing algorithms on prefixes)

Definition at line 357 of file read_binary.hpp.

Referenced by thrill::api::ReadBinary().

◆ ReadBinary() [2/4]

DIA<ValueType> thrill::api::ReadBinary	(	struct LocalStorageTag	,
		Context &	ctx,
		const std::vector< std::string > &	filepath,
		uint64_t	size_limit = `ReadBinaryNode<ValueType>::no_size_limit_`
	)

ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA.

Parameters

ctx	Reference to the context object
filepath	Path of the file in the file system
size_limit	Optional limit to the total file size (e.g. for testing algorithms on prefixes)

Definition at line 381 of file read_binary.hpp.

◆ ReadBinary() [3/4]

DIA<ValueType> thrill::api::ReadBinary	(	Context &	ctx,
		const std::string &	filepath,
		uint64_t	size_limit = `ReadBinaryNode<ValueType>::no_size_limit_`
	)

ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA.

Parameters

ctx	Reference to the context object
filepath	Path of the file in the file system
size_limit	Optional limit to the total file size (e.g. for testing algorithms on prefixes)

Definition at line 406 of file read_binary.hpp.

◆ ReadBinary() [4/4]

DIA<ValueType> thrill::api::ReadBinary	(	struct LocalStorageTag	,
		Context &	ctx,
		const std::string &	filepath,
		uint64_t	size_limit = `ReadBinaryNode<ValueType>::no_size_limit_`
	)

ReadBinary is a DOp, which reads a file written by WriteBinary from the file system and creates a DIA.

Parameters

ctx	Reference to the context object
filepath	Path of the file in the file system
size_limit	Optional limit to the total file size (e.g. for testing algorithms on prefixes)

Definition at line 430 of file read_binary.hpp.

References thrill::api::ReadBinary().

◆ ReadLines() [1/4]

DIA<std::string> thrill::api::ReadLines	(	Context &	ctx,
		const std::string &	filepath
	)

ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function.

Parameters

ctx	Reference to the context object
filepath	Path of the file in the file system

Examples:: examples/tutorial/k-means_step5.cpp, and examples/tutorial/k-means_step6.cpp.

Definition at line 454 of file read_lines.hpp.

Referenced by CountTrianglesPerLine(), JoinTPCH4(), LoadBFSGraph(), LoadPoints(), main(), Percentiles(), ReadInputFile(), thrill::api::ReadLines(), RunHashWordCount(), RunJoinPageRankEdgePerLine(), RunKMeansFile(), RunPageRankEdgePerLine(), RunStochasticGradFile(), RunWordCount(), and WordCount().

◆ ReadLines() [2/4]

DIA<std::string> thrill::api::ReadLines	(	struct LocalStorageTag	,
		Context &	ctx,
		const std::string &	filepath
	)

ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function.

Parameters

ctx	Reference to the context object
filepath	Path of the file in the file system

Definition at line 471 of file read_lines.hpp.

◆ ReadLines() [3/4]

DIA<std::string> thrill::api::ReadLines	(	Context &	ctx,
		const std::vector< std::string > &	filepaths
	)

ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function.

Parameters

ctx	Reference to the context object
filepaths	Path of the file in the file system

Definition at line 489 of file read_lines.hpp.

◆ ReadLines() [4/4]

DIA<std::string> thrill::api::ReadLines	(	struct LocalStorageTag	,
		Context &	ctx,
		const std::vector< std::string > &	filepaths
	)

ReadLines is a DOp, which reads a file from the file system and creates an ordered DIA according to a given read function.

Parameters

ctx	Reference to the context object
filepaths	Path of the file in the file system

Definition at line 507 of file read_lines.hpp.

References thrill::api::ReadLines().

Functions

Detailed Description

Function Documentation

◆ ConcatToDIA()

◆ Distribute() [1/2]

◆ Distribute() [2/2]

◆ EqualToDIA() [1/2]

◆ EqualToDIA() [2/2]

◆ Generate() [1/2]

◆ Generate() [2/2]

◆ ReadBinary() [1/4]

◆ ReadBinary() [2/4]

◆ ReadBinary() [3/4]

◆ ReadBinary() [4/4]

◆ ReadLines() [1/4]

◆ ReadLines() [2/4]

◆ ReadLines() [3/4]

◆ ReadLines() [4/4]