Detailed Description
Reader to retrieve items in unordered sequence from a MixBlockQueue.
This is not a full implementation of all methods available in a normal BlockReader. Mainly, this is because only retrieval of whole items are possible. Due to the unordered sequence, these probably have to be all of equal type as well.
The Reader supports all combinations of consuming and keeping. However, do not assume that the second round of reading delivers items in the same order as the first. This is because once items are cached inside the BlockQueues of MixBlockQueue, we use a plain CatReader to deliver them again (which is probably faster as it has a sequential access pattern).
See MixBlockQueue for more information on how items are read.
Definition at line 161 of file mix_block_queue.hpp.
Collaboration diagram for MixBlockQueueReader:#include <mix_block_queue.hpp>
Public Types | |
| using | CatBlockReader = BlockReader< CatBlockSource > |
| using | CatBlockSource = data::CatBlockSource< DynBlockSource > |
Public Member Functions | |
| MixBlockQueueReader (MixBlockQueue &mix_queue, bool consume, size_t local_worker_id) | |
| MixBlockQueueReader (const MixBlockQueueReader &)=delete | |
| non-copyable: delete copy-constructor More... | |
| MixBlockQueueReader (MixBlockQueueReader &&)=default | |
| move-constructor: default More... | |
| ~MixBlockQueueReader () | |
| Possibly consume unread blocks. More... | |
| bool | HasNext () |
| HasNext() returns true if at least one more item is available. More... | |
| template<typename T > | |
| T | Next () |
| Next() reads a complete item T. More... | |
| MixBlockQueueReader & | operator= (const MixBlockQueueReader &)=delete |
| non-copyable: delete assignment operator More... | |
| MixBlockQueueReader & | operator= (MixBlockQueueReader &&)=default |
| move-assignment operator: default More... | |
Private Member Functions | |
| bool | PullBlock () |
Private Attributes | |
| CatBlockReader | cat_reader_ { CatBlockSource() } |
| MixBlockQueue & | mix_queue_ |
| reference to mix queue More... | |
| const bool | reread_ |
Attributes for Mix Reading | |
| std::vector< BlockQueue::Reader > | readers_ |
| sub-readers for each block queue in mix queue More... | |
| size_t | selected_ = size_t(-1) |
| reader currently selected More... | |
| size_t | available_ = 0 |
| number of available items on the selected reader More... | |
| std::vector< size_t > | available_at_ |
| size_t | open_ = mix_queue_.num_workers_ |
| number of readers still open More... | |
Static Private Attributes | |
| static constexpr bool | debug = false |
Member Typedef Documentation
◆ CatBlockReader
| using CatBlockReader = BlockReader<CatBlockSource> |
Definition at line 167 of file mix_block_queue.hpp.
◆ CatBlockSource
Definition at line 166 of file mix_block_queue.hpp.
Constructor & Destructor Documentation
◆ MixBlockQueueReader() [1/3]
| MixBlockQueueReader | ( | MixBlockQueue & | mix_queue, |
| bool | consume, | ||
| size_t | local_worker_id | ||
| ) |
Definition at line 87 of file mix_block_queue.cpp.
References MixBlockQueueReader::available_at_, MixBlockQueueReader::cat_reader_, MixBlockQueueReader::mix_queue_, MixBlockQueue::num_workers_, MixBlockQueue::queues_, MixBlockQueueReader::readers_, and MixBlockQueueReader::reread_.
◆ MixBlockQueueReader() [2/3]
|
delete |
non-copyable: delete copy-constructor
◆ MixBlockQueueReader() [3/3]
|
default |
move-constructor: default
◆ ~MixBlockQueueReader()
| ~MixBlockQueueReader | ( | ) |
Possibly consume unread blocks.
Definition at line 113 of file mix_block_queue.cpp.
Member Function Documentation
◆ HasNext()
|
inline |
HasNext() returns true if at least one more item is available.
Definition at line 185 of file mix_block_queue.hpp.
Referenced by SortNode< ValueType, CompareFunction, SortAlgorithm, Stable >::MainOp(), and JoinNode< ValueType, FirstDIA, SecondDIA, KeyExtractor1, KeyExtractor2, JoinFunction, HashFunction, UseLocationDetection >::ReceiveItems().
◆ Next()
|
inline |
Next() reads a complete item T.
Definition at line 196 of file mix_block_queue.hpp.
◆ operator=() [1/2]
|
delete |
non-copyable: delete assignment operator
◆ operator=() [2/2]
|
default |
move-assignment operator: default
◆ PullBlock()
|
private |
Definition at line 117 of file mix_block_queue.cpp.
References MixBlockQueueReader::available_, MixBlockQueueReader::available_at_, MixBlockQueue::SrcBlockPair::block, Block::IsValid(), LOG, MixBlockQueueReader::mix_queue_, Block::num_items(), MixBlockQueueReader::open_, MixBlockQueue::Pop(), MixBlockQueue::queues_, MixBlockQueueReader::readers_, MixBlockQueueReader::selected_, and MixBlockQueue::SrcBlockPair::src.
Member Data Documentation
◆ available_
|
private |
number of available items on the selected reader
Definition at line 236 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::PullBlock().
◆ available_at_
|
private |
number of additional items available at reader (excluding current available_)
Definition at line 240 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::MixBlockQueueReader(), and MixBlockQueueReader::PullBlock().
◆ cat_reader_
|
private |
for rereading the mix queue: use a cat reader on the embedded BlockQueue's files.
Definition at line 249 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::MixBlockQueueReader().
◆ debug
|
staticprivate |
Definition at line 163 of file mix_block_queue.hpp.
◆ mix_queue_
|
private |
reference to mix queue
Definition at line 220 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::MixBlockQueueReader(), and MixBlockQueueReader::PullBlock().
◆ open_
|
private |
number of readers still open
Definition at line 243 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::PullBlock().
◆ readers_
|
private |
sub-readers for each block queue in mix queue
Definition at line 230 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::MixBlockQueueReader(), and MixBlockQueueReader::PullBlock().
◆ reread_
|
private |
flag whether we are rereading the mix queue by reading the files using a cat_reader_.
Definition at line 224 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::MixBlockQueueReader().
◆ selected_
|
private |
reader currently selected
Definition at line 233 of file mix_block_queue.hpp.
Referenced by MixBlockQueueReader::PullBlock().
The documentation for this class was generated from the following files:
- thrill/data/mix_block_queue.hpp
- thrill/data/mix_block_queue.cpp
