Thrill  0.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
User DIA API

Generate is a Source-DOp, which creates a DIA of given size using a generator function.

From the user's perspective, a DIA (Distributed Immutable Array) is an immutable array, which can only be accessed by Operations and Actions. Operations transform a DIA into another DIA, Actions aggregate data of the DIA and make it accessible to the user program.

Inside the Thrill framework, a DIA only consists of a reference to a DIANode and a lambda function, which can transform elements from the DIANode to this DIA. Distributed Operations(DOps) and Actions both need a global barrier and thus form a new DIANode. Every DOp and Action has an own node type, which are sub-classes of DIANode.

DIABase is the untyped super-class of DIANode. DIABase objects form a DAG, in which the parent-child relation means that a parent has to be executed before the child can be executed. Every DIANode can be executed, which performs the actual operation and all uncalculated parent operations. The engine uses the DIABase graph to optimize the order of operations.

List of Actions

Generate (ctx, generator_function, size)

Generate is a Source-DOp, which creates a DIA of given size using a generator function. The generator function called for each index in the range of [0,size) and must output exactly one item.

Parameters
ctxReference to the Context object
sizeSize of the output DIA
generate_functionGenerator function, which maps size_t from [0,size) to elements. Input type has to be size_t.

Distribute(ctx, vector)

Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA. Distribute is a Source DOp, which scatters the vector data from the source_id to all workers, partitioning equally, and returning the data in a DIA.