Thrill  0.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
file_io.hpp File Reference
#include <thrill/common/math.hpp>
#include <thrill/common/porting.hpp>
#include <thrill/common/system_exception.hpp>
#include <tlx/counting_ptr.hpp>
#include <string>
#include <vector>
+ Include dependency graph for file_io.hpp:
+ This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

struct  FileInfo
 General information of vfs file. More...
 
struct  FileList
 List of file info and additional overall info. More...
 
class  ReadStream
 Reader object from any source. More...
 
class  WriteStream
 Writer object to output data to any supported URI. More...
 

Namespaces

 thrill
 
 thrill::vfs
 

Macros

#define THRILL_VFS_FILE_IO_HEADER
 

Typedefs

using ReadStreamPtr = tlx::CountingPtr< ReadStream >
 
using WriteStreamPtr = tlx::CountingPtr< WriteStream >
 

Enumerations

enum  GlobType { All, File, Directory }
 Type of objects to include in glob result. More...
 
enum  Type { File, Directory }
 VFS object type. More...
 

Functions

void Deinitialize ()
 Deinitialize VFS layer. More...
 
std::string FillFilePattern (const std::string &pathbase, size_t worker, size_t file_part)
 
FileList Glob (const std::vector< std::string > &globlist, const GlobType &gtype=GlobType::All)
 Reads a glob path list and deliver a file list, sizes, and prefixsums (in bytes) for all matching files. More...
 
FileList Glob (const std::string &glob, const GlobType &gtype=GlobType::All)
 Reads a glob path list and deliver a file list, sizes, and prefixsums (in bytes) for all matching files. More...
 
void Initialize ()
 Initialize VFS layer. More...
 
bool IsCompressed (const std::string &path)
 
bool IsRemoteUri (const std::string &path)
 Returns true, if file at filepath is a remote uri like s3:// or hdfs://. More...
 
ReadStreamPtr OpenReadStream (const std::string &path, const common::Range &range=common::Range())
 Construct reader for given path uri. More...
 
WriteStreamPtr OpenWriteStream (const std::string &path)
 
std::ostream & operator<< (std::ostream &os, const Type &t)
 

Macro Definition Documentation

#define THRILL_VFS_FILE_IO_HEADER

Definition at line 16 of file file_io.hpp.