Thrill  0.1
MixBlockQueueReader Class Reference

Detailed Description

Reader to retrieve items in unordered sequence from a MixBlockQueue.

This is not a full implementation of all methods available in a normal BlockReader. Mainly, this is because only retrieval of whole items are possible. Due to the unordered sequence, these probably have to be all of equal type as well.

The Reader supports all combinations of consuming and keeping. However, do not assume that the second round of reading delivers items in the same order as the first. This is because once items are cached inside the BlockQueues of MixBlockQueue, we use a plain CatReader to deliver them again (which is probably faster as it has a sequential access pattern).

See MixBlockQueue for more information on how items are read.

Definition at line 161 of file mix_block_queue.hpp.

+ Collaboration diagram for MixBlockQueueReader:

#include <mix_block_queue.hpp>

Public Types

using CatBlockReader = BlockReader< CatBlockSource >
 
using CatBlockSource = data::CatBlockSource< DynBlockSource >
 

Public Member Functions

 MixBlockQueueReader (MixBlockQueue &mix_queue, bool consume, size_t local_worker_id)
 
 MixBlockQueueReader (const MixBlockQueueReader &)=delete
 non-copyable: delete copy-constructor More...
 
 MixBlockQueueReader (MixBlockQueueReader &&)=default
 move-constructor: default More...
 
 ~MixBlockQueueReader ()
 Possibly consume unread blocks. More...
 
bool HasNext ()
 HasNext() returns true if at least one more item is available. More...
 
template<typename T >
T Next ()
 Next() reads a complete item T. More...
 
MixBlockQueueReaderoperator= (const MixBlockQueueReader &)=delete
 non-copyable: delete assignment operator More...
 
MixBlockQueueReaderoperator= (MixBlockQueueReader &&)=default
 move-assignment operator: default More...
 

Private Member Functions

bool PullBlock ()
 

Private Attributes

CatBlockReader cat_reader_ { CatBlockSource() }
 
MixBlockQueuemix_queue_
 reference to mix queue More...
 
const bool reread_
 
Attributes for Mix Reading
std::vector< BlockQueue::Readerreaders_
 sub-readers for each block queue in mix queue More...
 
size_t selected_ = size_t(-1)
 reader currently selected More...
 
size_t available_ = 0
 number of available items on the selected reader More...
 
std::vector< size_t > available_at_
 
size_t open_ = mix_queue_.num_workers_
 number of readers still open More...
 

Static Private Attributes

static constexpr bool debug = false
 

Member Typedef Documentation

◆ CatBlockReader

Definition at line 167 of file mix_block_queue.hpp.

◆ CatBlockSource

Constructor & Destructor Documentation

◆ MixBlockQueueReader() [1/3]

◆ MixBlockQueueReader() [2/3]

non-copyable: delete copy-constructor

◆ MixBlockQueueReader() [3/3]

move-constructor: default

◆ ~MixBlockQueueReader()

Possibly consume unread blocks.

Definition at line 113 of file mix_block_queue.cpp.

Member Function Documentation

◆ HasNext()

◆ Next()

T Next ( )
inline

Next() reads a complete item T.

Definition at line 196 of file mix_block_queue.hpp.

◆ operator=() [1/2]

MixBlockQueueReader& operator= ( const MixBlockQueueReader )
delete

non-copyable: delete assignment operator

◆ operator=() [2/2]

MixBlockQueueReader& operator= ( MixBlockQueueReader &&  )
default

move-assignment operator: default

◆ PullBlock()

Member Data Documentation

◆ available_

size_t available_ = 0
private

number of available items on the selected reader

Definition at line 236 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::PullBlock().

◆ available_at_

std::vector<size_t> available_at_
private

number of additional items available at reader (excluding current available_)

Definition at line 240 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::MixBlockQueueReader(), and MixBlockQueueReader::PullBlock().

◆ cat_reader_

CatBlockReader cat_reader_ { CatBlockSource() }
private

for rereading the mix queue: use a cat reader on the embedded BlockQueue's files.

Definition at line 249 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::MixBlockQueueReader().

◆ debug

constexpr bool debug = false
staticprivate

Definition at line 163 of file mix_block_queue.hpp.

◆ mix_queue_

MixBlockQueue& mix_queue_
private

reference to mix queue

Definition at line 220 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::MixBlockQueueReader(), and MixBlockQueueReader::PullBlock().

◆ open_

size_t open_ = mix_queue_.num_workers_
private

number of readers still open

Definition at line 243 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::PullBlock().

◆ readers_

std::vector<BlockQueue::Reader> readers_
private

sub-readers for each block queue in mix queue

Definition at line 230 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::MixBlockQueueReader(), and MixBlockQueueReader::PullBlock().

◆ reread_

const bool reread_
private

flag whether we are rereading the mix queue by reading the files using a cat_reader_.

Definition at line 224 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::MixBlockQueueReader().

◆ selected_

size_t selected_ = size_t(-1)
private

reader currently selected

Definition at line 233 of file mix_block_queue.hpp.

Referenced by MixBlockQueueReader::PullBlock().


The documentation for this class was generated from the following files: