Thrill  0.1
ByteBlock Class Reference

Detailed Description

A ByteBlock is the basic storage units of containers like File, BlockQueue, etc.

It consists of a fixed number of bytes without any type and meta information. Conceptually a ByteBlock is written once and can then be shared read-only between containers using CountingPtr<const ByteBlock> reference counting inside a Block, which adds meta information.

ByteBlocks can be swapped to disk, which decreases their size to 0.

Definition at line 51 of file byte_block.hpp.

+ Inheritance diagram for ByteBlock:
+ Collaboration diagram for ByteBlock:

#include <byte_block.hpp>

Classes

class  Deleter
 deleter for CountingPtr<ByteBlock> More...
 

Public Types

using ByteBlockCPtr = tlx::CountingPtr< const ByteBlock, Deleter >
 
using ByteBlockPtr = tlx::CountingPtr< ByteBlock, Deleter >
 

Public Member Functions

Bytebegin ()
 mutable data accessor to beginning of memory block More...
 
const Bytebegin () const
 const data accessor to beginning of memory block More...
 
Bytedata ()
 mutable data accessor to memory block More...
 
const Bytedata () const
 const data accessor to memory block More...
 
void DecPinCount (size_t local_worker_id)
 decrement pin count, possibly signal block pool that if it reaches zero. More...
 
Byteend ()
 mutable data accessor beyond end of memory block More...
 
const Byteend () const
 const data accessor beyond end of memory block More...
 
bool has_ext_file () const
 Returns whether the ByteBlock is in an external file. More...
 
bool in_memory () const
 true if block resides in memory More...
 
void IncPinCount (size_t local_worker_id)
 increment pin count, must be >= 1 before. More...
 
bool is_deleted () const
 true if being deleted More...
 
size_t pin_count (size_t local_worker_id) const
 return current pin count More...
 
std::string pin_count_str () const
 return string list of pin_counts More...
 
size_t size () const
 the block size More...
 
- Public Member Functions inherited from ReferenceCounter
 ReferenceCounter () noexcept
 new objects have zero reference count More...
 
 ReferenceCounter (const ReferenceCounter &) noexcept
 coping still creates a new object with zero reference count More...
 
 ~ReferenceCounter ()
 
bool dec_reference () const noexcept
 Call whenever resetting (i.e. More...
 
void inc_reference () const noexcept
 Call whenever setting a pointer to the object. More...
 
ReferenceCounteroperator= (const ReferenceCounter &) noexcept
 assignment operator, leaves pointers unchanged More...
 
size_t reference_count () const noexcept
 Return the number of references to this object (for debugging) More...
 
bool unique () const noexcept
 Test if the ReferenceCounter is referenced by only one CountingPtr. More...
 

Private Member Functions

 ByteBlock ()=delete
 No default construction of Byteblock. More...
 
 ByteBlock (BlockPool *block_pool, Byte *data, size_t size)
 Constructor to initialize ByteBlock in a buffer of memory. More...
 
 ByteBlock (BlockPool *block_pool, const foxxll::file_ptr &ext_file, int64_t offset, size_t size)
 
void OnWriteComplete (foxxll::request *req, bool success)
 forwarded to block_pool_ More...
 

Private Attributes

BlockPoolblock_pool_
 reference to BlockPool for deletion. More...
 
Bytedata_
 
foxxll::BID< 0 > em_bid_
 
foxxll::file_ptr ext_file_
 
std::vector< size_t, mem::GPoolAllocator< size_t > > pin_count_
 counts the number of pins in this block per thread_id. More...
 
const size_t size_
 the allocated size of the buffer in bytes More...
 
size_t total_pins_ = 0
 

Static Private Attributes

static constexpr bool debug = false
 

Friends

std::ostream & operator<< (std::ostream &os, const ByteBlock &b)
 

Member Typedef Documentation

◆ ByteBlockCPtr

Definition at line 65 of file byte_block.hpp.

◆ ByteBlockPtr

Definition at line 64 of file byte_block.hpp.

Constructor & Destructor Documentation

◆ ByteBlock() [1/3]

ByteBlock ( )
privatedelete

No default construction of Byteblock.

◆ ByteBlock() [2/3]

ByteBlock ( BlockPool block_pool,
Byte data,
size_t  size 
)
private

Constructor to initialize ByteBlock in a buffer of memory.

Protected, used BlockPool::AllocateByteBlock() for construction.

Parameters
datathe memory address of the byte-blocks data. nullptr if swapped out
sizethe size of the block in bytes
block_poolthe block pool that manages this ByteBlock

Definition at line 29 of file byte_block.cpp.

◆ ByteBlock() [3/3]

ByteBlock ( BlockPool block_pool,
const foxxll::file_ptr ext_file,
int64_t  offset,
size_t  size 
)
private

Constructor to initialize ByteBlock as a mapping to an external foxxll::file area.

Definition at line 35 of file byte_block.cpp.

Member Function Documentation

◆ begin() [1/2]

Byte* begin ( )
inline

mutable data accessor to beginning of memory block

Definition at line 74 of file byte_block.hpp.

References ByteBlock::data_.

◆ begin() [2/2]

const Byte* begin ( ) const
inline

const data accessor to beginning of memory block

Definition at line 76 of file byte_block.hpp.

References ByteBlock::data_.

◆ data() [1/2]

Byte* data ( )
inline

mutable data accessor to memory block

Definition at line 69 of file byte_block.hpp.

References ByteBlock::data_.

◆ data() [2/2]

const Byte* data ( ) const
inline

const data accessor to memory block

Definition at line 71 of file byte_block.hpp.

References ByteBlock::data_.

◆ DecPinCount()

void DecPinCount ( size_t  local_worker_id)

decrement pin count, possibly signal block pool that if it reaches zero.

Definition at line 71 of file byte_block.cpp.

References ByteBlock::block_pool_, and BlockPool::DecBlockPinCount().

Referenced by ByteBlock::is_deleted(), PinnedByteBlockPtr::operator=(), PinnedByteBlockPtr::ReleasePin(), and PinnedByteBlockPtr::~PinnedByteBlockPtr().

◆ end() [1/2]

Byte* end ( )
inline

mutable data accessor beyond end of memory block

Definition at line 79 of file byte_block.hpp.

References ByteBlock::data_, and ByteBlock::size_.

◆ end() [2/2]

const Byte* end ( ) const
inline

const data accessor beyond end of memory block

Definition at line 81 of file byte_block.hpp.

References ByteBlock::data_, and ByteBlock::size_.

◆ has_ext_file()

bool has_ext_file ( ) const
inline

Returns whether the ByteBlock is in an external file.

Definition at line 87 of file byte_block.hpp.

References ByteBlock::ext_file_, and CountingPtr< Type, Deleter >::get().

◆ in_memory()

bool in_memory ( ) const
inline

true if block resides in memory

Definition at line 98 of file byte_block.hpp.

References ByteBlock::data_.

Referenced by BlockPool::EvictBlock(), and BlockPool::PinBlock().

◆ IncPinCount()

void IncPinCount ( size_t  local_worker_id)

increment pin count, must be >= 1 before.

Definition at line 67 of file byte_block.cpp.

References ByteBlock::block_pool_, and BlockPool::IncBlockPinCount().

Referenced by ByteBlock::is_deleted(), and PinnedByteBlockPtr::PinnedByteBlockPtr().

◆ is_deleted()

bool is_deleted ( ) const
inline

true if being deleted

Definition at line 103 of file byte_block.hpp.

References ByteBlock::DecPinCount(), ByteBlock::IncPinCount(), and ByteBlock::pin_count_.

Referenced by BlockPool::OnWriteComplete().

◆ OnWriteComplete()

void OnWriteComplete ( foxxll::request req,
bool  success 
)
private

forwarded to block_pool_

Definition at line 75 of file byte_block.cpp.

References ByteBlock::block_pool_, and BlockPool::OnWriteComplete().

Referenced by BlockPool::EvictBlockLRU().

◆ pin_count()

size_t pin_count ( size_t  local_worker_id) const
inline

return current pin count

Definition at line 90 of file byte_block.hpp.

References ByteBlock::pin_count_, and ByteBlock::pin_count_str().

Referenced by BlockPool::DecBlockPinCount().

◆ pin_count_str()

std::string pin_count_str ( ) const

return string list of pin_counts

Definition at line 63 of file byte_block.cpp.

References tlx::join(), and ByteBlock::pin_count_.

Referenced by ByteBlock::Deleter::operator()(), and ByteBlock::pin_count().

◆ size()

Friends And Related Function Documentation

◆ operator<<

std::ostream& operator<< ( std::ostream &  os,
const ByteBlock b 
)
friend

Definition at line 79 of file byte_block.cpp.

Member Data Documentation

◆ block_pool_

◆ data_

Byte* data_
private

the memory block itself is referenced as it is in a a separate memory region that can be swapped out

Definition at line 116 of file byte_block.hpp.

Referenced by ByteBlock::begin(), ByteBlock::data(), ByteBlock::end(), BlockPool::EvictBlockLRU(), ByteBlock::in_memory(), BlockPool::OnWriteComplete(), and thrill::data::operator<<().

◆ debug

constexpr bool debug = false
staticprivate

Definition at line 53 of file byte_block.hpp.

◆ em_bid_

foxxll::BID<0> em_bid_
private

external memory block, which contains a pointer to foxxll::file, an offset into the file, and (unfortunately) also the size.

Definition at line 133 of file byte_block.hpp.

Referenced by BlockPool::EvictBlockLRU(), BlockPool::OnReadComplete(), BlockPool::OnWriteComplete(), and BlockPool::PinBlock().

◆ ext_file_

foxxll::file_ptr ext_file_
private

shared pointer to external file, if this is != nullptr then the Block was created for directly reading binary files.

Definition at line 137 of file byte_block.hpp.

Referenced by BlockPool::EvictBlockLRU(), ByteBlock::has_ext_file(), BlockPool::OnReadComplete(), BlockPool::OnWriteComplete(), thrill::data::operator<<(), and BlockPool::PinBlock().

◆ pin_count_

std::vector<size_t, mem::GPoolAllocator<size_t> > pin_count_
private

◆ size_

const size_t size_
private

the allocated size of the buffer in bytes

Definition at line 119 of file byte_block.hpp.

Referenced by ByteBlock::end(), thrill::data::operator<<(), and ByteBlock::size().

◆ total_pins_

size_t total_pins_ = 0
private

counts the total number of pins, the data_ may be swapped out when this reaches zero.

Definition at line 129 of file byte_block.hpp.

Referenced by BlockPool::DecBlockPinCount(), BlockPool::IntIncBlockPinCount(), ByteBlock::Deleter::operator()(), thrill::data::operator<<(), and BlockPool::PinBlock().


The documentation for this class was generated from the following files: