Thrill
0.1
|
A ByteBlock is the basic storage units of containers like File, BlockQueue, etc.
It consists of a fixed number of bytes without any type and meta information. Conceptually a ByteBlock is written once and can then be shared read-only between containers using CountingPtr<const ByteBlock> reference counting inside a Block, which adds meta information.
ByteBlocks can be swapped to disk, which decreases their size to 0.
Definition at line 51 of file byte_block.hpp.
#include <byte_block.hpp>
Classes | |
class | Deleter |
deleter for CountingPtr<ByteBlock> More... | |
Public Types | |
using | ByteBlockCPtr = tlx::CountingPtr< const ByteBlock, Deleter > |
using | ByteBlockPtr = tlx::CountingPtr< ByteBlock, Deleter > |
Public Member Functions | |
Byte * | begin () |
mutable data accessor to beginning of memory block More... | |
const Byte * | begin () const |
const data accessor to beginning of memory block More... | |
Byte * | data () |
mutable data accessor to memory block More... | |
const Byte * | data () const |
const data accessor to memory block More... | |
void | DecPinCount (size_t local_worker_id) |
decrement pin count, possibly signal block pool that if it reaches zero. More... | |
Byte * | end () |
mutable data accessor beyond end of memory block More... | |
const Byte * | end () const |
const data accessor beyond end of memory block More... | |
bool | has_ext_file () const |
Returns whether the ByteBlock is in an external file. More... | |
bool | in_memory () const |
true if block resides in memory More... | |
void | IncPinCount (size_t local_worker_id) |
increment pin count, must be >= 1 before. More... | |
bool | is_deleted () const |
true if being deleted More... | |
size_t | pin_count (size_t local_worker_id) const |
return current pin count More... | |
std::string | pin_count_str () const |
return string list of pin_counts More... | |
size_t | size () const |
the block size More... | |
Public Member Functions inherited from ReferenceCounter | |
ReferenceCounter () noexcept | |
new objects have zero reference count More... | |
ReferenceCounter (const ReferenceCounter &) noexcept | |
coping still creates a new object with zero reference count More... | |
~ReferenceCounter () | |
bool | dec_reference () const noexcept |
Call whenever resetting (i.e. More... | |
void | inc_reference () const noexcept |
Call whenever setting a pointer to the object. More... | |
ReferenceCounter & | operator= (const ReferenceCounter &) noexcept |
assignment operator, leaves pointers unchanged More... | |
size_t | reference_count () const noexcept |
Return the number of references to this object (for debugging) More... | |
bool | unique () const noexcept |
Test if the ReferenceCounter is referenced by only one CountingPtr. More... | |
Private Member Functions | |
ByteBlock ()=delete | |
No default construction of Byteblock. More... | |
ByteBlock (BlockPool *block_pool, Byte *data, size_t size) | |
Constructor to initialize ByteBlock in a buffer of memory. More... | |
ByteBlock (BlockPool *block_pool, const foxxll::file_ptr &ext_file, int64_t offset, size_t size) | |
void | OnWriteComplete (foxxll::request *req, bool success) |
forwarded to block_pool_ More... | |
Private Attributes | |
BlockPool * | block_pool_ |
reference to BlockPool for deletion. More... | |
Byte * | data_ |
foxxll::BID< 0 > | em_bid_ |
foxxll::file_ptr | ext_file_ |
std::vector< size_t, mem::GPoolAllocator< size_t > > | pin_count_ |
counts the number of pins in this block per thread_id. More... | |
const size_t | size_ |
the allocated size of the buffer in bytes More... | |
size_t | total_pins_ = 0 |
Static Private Attributes | |
static constexpr bool | debug = false |
Friends | |
std::ostream & | operator<< (std::ostream &os, const ByteBlock &b) |
using ByteBlockCPtr = tlx::CountingPtr<const ByteBlock, Deleter> |
Definition at line 65 of file byte_block.hpp.
using ByteBlockPtr = tlx::CountingPtr<ByteBlock, Deleter> |
Definition at line 64 of file byte_block.hpp.
|
privatedelete |
No default construction of Byteblock.
Constructor to initialize ByteBlock in a buffer of memory.
Protected, used BlockPool::AllocateByteBlock() for construction.
data | the memory address of the byte-blocks data. nullptr if swapped out |
size | the size of the block in bytes |
block_pool | the block pool that manages this ByteBlock |
Definition at line 29 of file byte_block.cpp.
|
private |
Constructor to initialize ByteBlock as a mapping to an external foxxll::file area.
Definition at line 35 of file byte_block.cpp.
|
inline |
mutable data accessor to beginning of memory block
Definition at line 74 of file byte_block.hpp.
References ByteBlock::data_.
|
inline |
const data accessor to beginning of memory block
Definition at line 76 of file byte_block.hpp.
References ByteBlock::data_.
|
inline |
mutable data accessor to memory block
Definition at line 69 of file byte_block.hpp.
References ByteBlock::data_.
|
inline |
const data accessor to memory block
Definition at line 71 of file byte_block.hpp.
References ByteBlock::data_.
void DecPinCount | ( | size_t | local_worker_id | ) |
decrement pin count, possibly signal block pool that if it reaches zero.
Definition at line 71 of file byte_block.cpp.
References ByteBlock::block_pool_, and BlockPool::DecBlockPinCount().
Referenced by ByteBlock::is_deleted(), PinnedByteBlockPtr::operator=(), PinnedByteBlockPtr::ReleasePin(), and PinnedByteBlockPtr::~PinnedByteBlockPtr().
|
inline |
mutable data accessor beyond end of memory block
Definition at line 79 of file byte_block.hpp.
References ByteBlock::data_, and ByteBlock::size_.
|
inline |
const data accessor beyond end of memory block
Definition at line 81 of file byte_block.hpp.
References ByteBlock::data_, and ByteBlock::size_.
|
inline |
Returns whether the ByteBlock is in an external file.
Definition at line 87 of file byte_block.hpp.
References ByteBlock::ext_file_, and CountingPtr< Type, Deleter >::get().
|
inline |
true if block resides in memory
Definition at line 98 of file byte_block.hpp.
References ByteBlock::data_.
Referenced by BlockPool::EvictBlock(), and BlockPool::PinBlock().
void IncPinCount | ( | size_t | local_worker_id | ) |
increment pin count, must be >= 1 before.
Definition at line 67 of file byte_block.cpp.
References ByteBlock::block_pool_, and BlockPool::IncBlockPinCount().
Referenced by ByteBlock::is_deleted(), and PinnedByteBlockPtr::PinnedByteBlockPtr().
|
inline |
true if being deleted
Definition at line 103 of file byte_block.hpp.
References ByteBlock::DecPinCount(), ByteBlock::IncPinCount(), and ByteBlock::pin_count_.
Referenced by BlockPool::OnWriteComplete().
|
private |
forwarded to block_pool_
Definition at line 75 of file byte_block.cpp.
References ByteBlock::block_pool_, and BlockPool::OnWriteComplete().
Referenced by BlockPool::EvictBlockLRU().
|
inline |
return current pin count
Definition at line 90 of file byte_block.hpp.
References ByteBlock::pin_count_, and ByteBlock::pin_count_str().
Referenced by BlockPool::DecBlockPinCount().
std::string pin_count_str | ( | ) | const |
return string list of pin_counts
Definition at line 63 of file byte_block.cpp.
References tlx::join(), and ByteBlock::pin_count_.
Referenced by ByteBlock::Deleter::operator()(), and ByteBlock::pin_count().
|
inline |
the block size
Definition at line 84 of file byte_block.hpp.
References ByteBlock::size_.
Referenced by BlockPool::DecBlockPinCount(), BlockPool::EvictBlock(), BlockPool::EvictBlockLRU(), BlockPool::OnReadComplete(), BlockPool::OnWriteComplete(), and BlockPool::PinBlock().
|
friend |
Definition at line 79 of file byte_block.cpp.
|
private |
reference to BlockPool for deletion.
Definition at line 122 of file byte_block.hpp.
Referenced by ByteBlock::DecPinCount(), ByteBlock::IncPinCount(), ByteBlock::OnWriteComplete(), ByteBlock::Deleter::operator()(), and thrill::data::operator<<().
|
private |
the memory block itself is referenced as it is in a a separate memory region that can be swapped out
Definition at line 116 of file byte_block.hpp.
Referenced by ByteBlock::begin(), ByteBlock::data(), ByteBlock::end(), BlockPool::EvictBlockLRU(), ByteBlock::in_memory(), BlockPool::OnWriteComplete(), and thrill::data::operator<<().
|
staticprivate |
Definition at line 53 of file byte_block.hpp.
|
private |
external memory block, which contains a pointer to foxxll::file, an offset into the file, and (unfortunately) also the size.
Definition at line 133 of file byte_block.hpp.
Referenced by BlockPool::EvictBlockLRU(), BlockPool::OnReadComplete(), BlockPool::OnWriteComplete(), and BlockPool::PinBlock().
|
private |
shared pointer to external file, if this is != nullptr then the Block was created for directly reading binary files.
Definition at line 137 of file byte_block.hpp.
Referenced by BlockPool::EvictBlockLRU(), ByteBlock::has_ext_file(), BlockPool::OnReadComplete(), BlockPool::OnWriteComplete(), thrill::data::operator<<(), and BlockPool::PinBlock().
|
private |
counts the number of pins in this block per thread_id.
Definition at line 125 of file byte_block.hpp.
Referenced by BlockPool::DecBlockPinCount(), BlockPool::IncBlockPinCount(), BlockPool::IntIncBlockPinCount(), ByteBlock::is_deleted(), ByteBlock::pin_count(), ByteBlock::pin_count_str(), and BlockPool::PinBlock().
|
private |
the allocated size of the buffer in bytes
Definition at line 119 of file byte_block.hpp.
Referenced by ByteBlock::end(), thrill::data::operator<<(), and ByteBlock::size().
|
private |
counts the total number of pins, the data_ may be swapped out when this reaches zero.
Definition at line 129 of file byte_block.hpp.
Referenced by BlockPool::DecBlockPinCount(), BlockPool::IntIncBlockPinCount(), ByteBlock::Deleter::operator()(), thrill::data::operator<<(), and BlockPool::PinBlock().