XRootD
Loading...
Searching...
No Matches
XrdPfc::Info Class Reference

Status of cached file. Can be read from and written into a binary file. More...

#include <XrdPfcInfo.hh>

+ Collaboration diagram for XrdPfc::Info:

Classes

struct  AStat
 Access statistics. More...
 
struct  Status
 
union  Status.__unnamed127__
 
struct  Status.__unnamed127__.__unnamed129__
 
struct  Store
 

Public Member Functions

 Info (XrdSysTrace *trace, bool prefetchBuffer=false)
 Constructor.
 
 ~Info ()
 Destructor.
 
void CalcCksumMd5 (unsigned char *buff, char *digest)
 
uint32_t CalcCksumStore ()
 Get cksum, MD5 is for backward compatibility with V2 and V3.
 
uint32_t CalcCksumSyncedAndAStats ()
 
void CompactifyAccessRecords ()
 Compactify access records to the configured maximum.
 
int CountBlocksNotWrittenInRng (int firstIdx, int lastIdx) const
 Check download status in given block range.
 
void DowngradeCkSumState (CkSumCheck_e css_ref)
 
size_t GetAccessCnt () const
 Get number of accesses.
 
int GetBitvecSizeInBytes () const
 Get size of download-state bit-vector in bytes.
 
long long GetBufferSize () const
 Get prefetch buffer size.
 
CkSumCheck_e GetCkSumState () const
 
const char * GetCkSumStateAsText () const
 
time_t GetCreationTime () const
 Get file size.
 
long long GetExpectedDataFileSize () const
 Get expected data file size.
 
long long GetFileSize () const
 Get file size.
 
const AStatGetLastAccessStats () const
 Get latest access stats.
 
int GetLastDownloadedBlock () const
 Get number of the last downloaded block.
 
bool GetLatestDetachTime (time_t &t) const
 Get latest detach time.
 
int GetNBlocks () const
 Get number of blocks represented in download-state bit-vector.
 
int GetNDownloadedBlocks () const
 Get number of downloaded blocks.
 
long long GetNDownloadedBytes () const
 Get number of downloaded bytes.
 
time_t GetNoCkSumTime () const
 
time_t GetNoCkSumTimeForUVKeep () const
 
XrdSysTraceGetTrace () const
 
int GetVersion ()
 Get version.
 
bool HasNoCkSumTime () const
 
bool IsCkSumAny () const
 
bool IsCkSumBoth () const
 
bool IsCkSumCache () const
 
bool IsCkSumNet () const
 
bool IsComplete () const
 Get complete status.
 
bool Read (XrdOssDF *fp, const char *dname, const char *fname=0)
 Read content of cinfo file into this object.
 
const std::vector< AStat > & RefAStats () const
 
const StoreRefStoredData () const
 Get stored data.
 
void ResetAllAccessStats ()
 Reset IO Stats.
 
void ResetCkSumCache ()
 
void ResetCkSumNet ()
 
void ResetNoCkSumTime ()
 
void ResizeBits ()
 Reserve bit vectors for file_size / buffer_size bytes.
 
void SetAllBitsSynced ()
 Mark all blocks as synced to disk.
 
void SetBitPrefetch (int i)
 Mark block as obtained through prefetch.
 
void SetBitSynced (int i)
 Mark block as synced to disk.
 
void SetBitWritten (int i)
 Mark block as written to disk.
 
void SetBufferSizeFileSizeAndCreationTime (long long bs, long long fs)
 
void SetCkSumState (CkSumCheck_e css)
 
bool TestBitPrefetch (int i) const
 Test if block at the given index has been prefetched.
 
bool TestBitWritten (int i) const
 Test if block at the given index is written to disk.
 
void UpdateDownloadCompleteStatus ()
 Update complete status.
 
bool Write (XrdOssDF *fp, const char *dname, const char *fname=0)
 
void WriteIOStat (Stats &s)
 Write bytes missed, hits, and disk.
 
void WriteIOStatAttach ()
 Write open time in the last entry of access statistics.
 
void WriteIOStatDetach (Stats &s)
 Write close time together with bytes missed, hits, and disk.
 
void WriteIOStatSingle (long long bytes_disk)
 Write single open/close time for given bytes read from disk.
 
void WriteIOStatSingle (long long bytes_disk, time_t att, time_t dtc)
 Write open/close with given time and bytes read from disk.
 

Static Public Attributes

static const char * m_traceID = "CInfo"
 
static const int s_defaultVersion = 4
 
static const char * s_infoExtension = ".cinfo"
 
static const size_t s_infoExtensionLen = strlen(Info::s_infoExtension)
 
static size_t s_maxNumAccess = 20
 

Protected Attributes

std::vector< AStatm_astats
 access records
 
int m_bitvecSizeInBits
 cached
 
unsigned char * m_buff_prefetch
 prefetch statistics
 
unsigned char * m_buff_synced
 disk written state vector
 
unsigned char * m_buff_written
 download state vector
 
bool m_complete
 cached; if false, set to true when missingBlocks hit zero
 
bool m_hasPrefetchBuffer
 constains current prefetch score
 
int m_missingBlocks
 cached, updated in SetBitWritten()
 
Store m_store
 
XrdSysTracem_trace
 
int m_version
 

Detailed Description

Status of cached file. Can be read from and written into a binary file.

Definition at line 40 of file XrdPfcInfo.hh.


Class Documentation

◆ XrdPfc::Info::Status.__unnamed127__

union XrdPfc::Info::Status.__unnamed127__

Definition at line 44 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed127__:
Class Members
struct Status.__unnamed127__.__unnamed129__ __unnamed__
unsigned int _raw_

◆ XrdPfc::Info::Status.__unnamed127__.__unnamed129__

struct XrdPfc::Info::Status.__unnamed127__.__unnamed129__

Definition at line 45 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed127__.__unnamed129__:
Class Members
int _free_bits_: 29
int f_cksum_check: 3 as in enum CkSumCheck_e

Constructor & Destructor Documentation

◆ Info()

Info::Info ( XrdSysTrace trace,
bool  prefetchBuffer = false 
)

Constructor.

Definition at line 123 of file XrdPfcInfo.cc.

123 :
124 m_trace(trace),
126 m_version(0),
129 m_complete(false),
130 m_hasPrefetchBuffer(prefetchBuffer),
131 m_cksCalcMd5(0)
132{}
unsigned char * m_buff_prefetch
prefetch statistics
unsigned char * m_buff_written
download state vector
unsigned char * m_buff_synced
disk written state vector
bool m_complete
cached; if false, set to true when missingBlocks hit zero
int m_missingBlocks
cached, updated in SetBitWritten()
int m_bitvecSizeInBits
cached
XrdSysTrace * m_trace
bool m_hasPrefetchBuffer
constains current prefetch score

◆ ~Info()

Info::~Info ( )

Destructor.

Definition at line 134 of file XrdPfcInfo.cc.

135{
136 if (m_buff_synced) free(m_buff_synced);
139 delete m_cksCalcMd5;
140}

References m_buff_prefetch, m_buff_synced, and m_buff_written.

Member Function Documentation

◆ CalcCksumMd5()

void Info::CalcCksumMd5 ( unsigned char *  buff,
char *  digest 
)

Definition at line 238 of file XrdPfcInfo.cc.

239{
240 if (m_cksCalcMd5)
241 m_cksCalcMd5->Init();
242 else
243 m_cksCalcMd5 = new XrdCksCalcmd5();
244
245 m_cksCalcMd5->Update((const char*)buff, GetBitvecSizeInBytes());
246 memcpy(digest, m_cksCalcMd5->Final(), 16);
247}
virtual void Update(const char *Buff, int BLen)=0
virtual void Init()=0
virtual char * Final()=0
int GetBitvecSizeInBytes() const
Get size of download-state bit-vector in bytes.

References XrdCksCalc::Final(), GetBitvecSizeInBytes(), XrdCksCalc::Init(), and XrdCksCalc::Update().

+ Here is the call graph for this function:

◆ CalcCksumStore()

uint32_t Info::CalcCksumStore ( )

Get cksum, MD5 is for backward compatibility with V2 and V3.

Definition at line 227 of file XrdPfcInfo.cc.

228{
229 return crc32c(0, &m_store, sizeof(Store));
230}
uint32_t crc32c(uint32_t crc, void const *buf, size_t len)

References crc32c(), and m_store.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CalcCksumSyncedAndAStats()

uint32_t Info::CalcCksumSyncedAndAStats ( )

Definition at line 232 of file XrdPfcInfo.cc.

233{
234 uint32_t cks = crc32c(0, m_buff_synced, GetBitvecSizeInBytes());
235 return crc32c(cks, m_astats.data(), m_astats.size() * sizeof(AStat));
236}
std::vector< AStat > m_astats
access records

References crc32c(), GetBitvecSizeInBytes(), m_astats, and m_buff_synced.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CompactifyAccessRecords()

void Info::CompactifyAccessRecords ( )

Compactify access records to the configured maximum.

Definition at line 379 of file XrdPfcInfo.cc.

380{
381 time_t now = time(0);
382
383 std::vector<AStat> &v = m_astats;
384
385 for (int i = 0; i < (int) v.size() - 1; ++i)
386 {
387 if (v[i].DetachTime == 0)
388 v[i].DetachTime = std::min(v[i].AttachTime + v[i].Duration / v[i].NumIos, v[i+1].AttachTime);
389 }
390
391 while (v.size() > s_maxNumAccess)
392 {
393 double min_s = 1e10;
394 int min_i = -1;
395
396 int M = (int) v.size() - 2;
397 for (int i = 0; i < M; ++i)
398 {
399 AStat &a = v[i], &b = v[i + 1];
400
401 time_t t = std::max((time_t) 1, (now - b.AttachTime) / 2 + (now - a.DetachTime) / 2);
402 double s = (double) (b.AttachTime - a.DetachTime) / t;
403
404 if (s < min_s)
405 {
406 min_s = s;
407 min_i = i;
408 }
409 }
410 assert(min_i != -1);
411
412 v[min_i].MergeWith(v[min_i + 1]);
413
414 v.erase(v.begin() + (min_i + 1));
415 }
416}
static size_t s_maxNumAccess

References XrdPfc::Info::AStat::DetachTime, m_astats, and s_maxNumAccess.

Referenced by Write().

+ Here is the caller graph for this function:

◆ CountBlocksNotWrittenInRng()

int XrdPfc::Info::CountBlocksNotWrittenInRng ( int  firstIdx,
int  lastIdx 
) const
inline

Check download status in given block range.

Definition at line 452 of file XrdPfcInfo.hh.

453{
454 // TODO rewrite to use full byte comparisons outside of edges ?
455 // Also, it seems to be always called with firstIdx = 0, lastIdx = m_bitvecSizeInBits.
456 int cnt = 0;
457 for (int i = firstIdx; i < lastIdx; ++i)
458 if (! TestBitWritten(i)) ++cnt;
459
460 return cnt;
461}
bool TestBitWritten(int i) const
Test if block at the given index is written to disk.

References TestBitWritten().

Referenced by UpdateDownloadCompleteStatus().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ DowngradeCkSumState()

void XrdPfc::Info::DowngradeCkSumState ( CkSumCheck_e  css_ref)
inline

Definition at line 295 of file XrdPfcInfo.hh.

295{ m_store.m_status.f_cksum_check &= css_ref; }
Status m_status
status information
Definition XrdPfcInfo.hh:83

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetAccessCnt()

size_t XrdPfc::Info::GetAccessCnt ( ) const
inline

Get number of accesses.

Definition at line 261 of file XrdPfcInfo.hh.

261{ return m_store.m_accessCnt; }
size_t m_accessCnt
total access count for the file
Definition XrdPfcInfo.hh:82

References XrdPfc::Info::Store::m_accessCnt, and m_store.

Referenced by XrdPfc::File::GetAccessCnt().

+ Here is the caller graph for this function:

◆ GetBitvecSizeInBytes()

int XrdPfc::Info::GetBitvecSizeInBytes ( ) const
inline

Get size of download-state bit-vector in bytes.

Definition at line 429 of file XrdPfcInfo.hh.

430{
432 return ((m_bitvecSizeInBits - 1)/8 + 1);
433 else
434 return 0;
435}

References m_bitvecSizeInBits.

Referenced by CalcCksumMd5(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitPrefetch(), SetBitSynced(), SetBitWritten(), TestBitPrefetch(), TestBitWritten(), and Write().

+ Here is the caller graph for this function:

◆ GetBufferSize()

long long XrdPfc::Info::GetBufferSize ( ) const
inline

Get prefetch buffer size.

Definition at line 469 of file XrdPfcInfo.hh.

470{
471 return m_store.m_buffer_size;
472}
long long m_buffer_size
buffer / block size
Definition XrdPfcInfo.hh:78

References XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::File::GetBlockSize().

+ Here is the caller graph for this function:

◆ GetCkSumState()

CkSumCheck_e XrdPfc::Info::GetCkSumState ( ) const
inline

Definition at line 286 of file XrdPfcInfo.hh.

286{ return (CkSumCheck_e) m_store.m_status.f_cksum_check; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetCkSumStateAsText()

const char * Info::GetCkSumStateAsText ( ) const

Definition at line 249 of file XrdPfcInfo.cc.

250{
251 switch (m_store.m_status.f_cksum_check) {
252 case CSChk_None : return "none";
253 case CSChk_Cache : return "cache";
254 case CSChk_Net : return "net";
255 case CSChk_Both : return "both";
256 default : return "unknown";
257 }
258}
@ CSChk_Cache

References XrdPfc::CSChk_Both, XrdPfc::CSChk_Cache, XrdPfc::CSChk_Net, XrdPfc::CSChk_None, XrdPfc::Info::Store::m_status, and m_store.

◆ GetCreationTime()

time_t XrdPfc::Info::GetCreationTime ( ) const
inline

Get file size.

Definition at line 277 of file XrdPfcInfo.hh.

277{ return m_store.m_creationTime; }
time_t m_creationTime
time the info file was created
Definition XrdPfcInfo.hh:80

References XrdPfc::Info::Store::m_creationTime, and m_store.

◆ GetExpectedDataFileSize()

long long XrdPfc::Info::GetExpectedDataFileSize ( ) const
inline

Get expected data file size.

Definition at line 420 of file XrdPfcInfo.hh.

421{
422 int last_block = GetLastDownloadedBlock();
423 if (last_block == m_bitvecSizeInBits - 1)
424 return m_store.m_file_size;
425 else
426 return (last_block + 1) * m_store.m_buffer_size;
427}
int GetLastDownloadedBlock() const
Get number of the last downloaded block.
long long m_file_size
size of file in bytes
Definition XrdPfcInfo.hh:79

References GetLastDownloadedBlock(), m_bitvecSizeInBits, XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_file_size, and m_store.

+ Here is the call graph for this function:

◆ GetFileSize()

long long XrdPfc::Info::GetFileSize ( ) const
inline

Get file size.

Definition at line 442 of file XrdPfcInfo.hh.

443{
444 return m_store.m_file_size;
445}

References XrdPfc::Info::Store::m_file_size, and m_store.

Referenced by XrdPfc::Cache::DetermineFullFileSize(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ GetLastAccessStats()

const Info::AStat * Info::GetLastAccessStats ( ) const

Get latest access stats.

Definition at line 489 of file XrdPfcInfo.cc.

490{
491 return m_astats.empty() ? 0 : & m_astats.back();
492}

References m_astats.

Referenced by XrdPfc::File::GetLastAccessStats().

+ Here is the caller graph for this function:

◆ GetLastDownloadedBlock()

int XrdPfc::Info::GetLastDownloadedBlock ( ) const
inline

Get number of the last downloaded block.

Definition at line 412 of file XrdPfcInfo.hh.

413{
414 for (int i = m_bitvecSizeInBits - 1; i >= 0; --i)
415 if (TestBitWritten(i)) return i;
416
417 return -1;
418}

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by GetExpectedDataFileSize().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetLatestDetachTime()

bool Info::GetLatestDetachTime ( time_t &  t) const

Get latest detach time.

Definition at line 470 of file XrdPfcInfo.cc.

471{
472 if (m_astats.empty())
473 {
475 }
476 else
477 {
478 const AStat& ls = m_astats.back();
479
480 if (ls.DetachTime == 0)
481 t = ls.AttachTime + ls.Duration;
482 else
483 t = ls.DetachTime;
484 }
485
486 return t != 0;
487}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, m_astats, XrdPfc::Info::Store::m_creationTime, and m_store.

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the caller graph for this function:

◆ GetNBlocks()

int XrdPfc::Info::GetNBlocks ( ) const
inline

Get number of blocks represented in download-state bit-vector.

Definition at line 437 of file XrdPfcInfo.hh.

438{
439 return m_bitvecSizeInBits;
440}

References m_bitvecSizeInBits.

Referenced by XrdPfc::File::GetNBlocks().

+ Here is the caller graph for this function:

◆ GetNDownloadedBlocks()

int XrdPfc::Info::GetNDownloadedBlocks ( ) const
inline

Get number of downloaded blocks.

Definition at line 398 of file XrdPfcInfo.hh.

399{
400 int cntd = 0;
401 for (int i = 0; i < m_bitvecSizeInBits; ++i)
402 if (TestBitWritten(i)) cntd++;
403
404 return cntd;
405}

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by XrdPfc::File::GetNDownloadedBlocks(), and GetNDownloadedBytes().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNDownloadedBytes()

long long XrdPfc::Info::GetNDownloadedBytes ( ) const
inline

Get number of downloaded bytes.

Definition at line 407 of file XrdPfcInfo.hh.

408{
410}
int GetNDownloadedBlocks() const
Get number of downloaded blocks.

References GetNDownloadedBlocks(), XrdPfc::Info::Store::m_buffer_size, and m_store.

+ Here is the call graph for this function:

◆ GetNoCkSumTime()

time_t XrdPfc::Info::GetNoCkSumTime ( ) const
inline

Definition at line 300 of file XrdPfcInfo.hh.

300{ return m_store.m_noCkSumTime; }
time_t m_noCkSumTime
time when first non-cksummed block was detected
Definition XrdPfcInfo.hh:81

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ GetNoCkSumTimeForUVKeep()

time_t XrdPfc::Info::GetNoCkSumTimeForUVKeep ( ) const
inline

◆ GetTrace()

XrdSysTrace * XrdPfc::Info::GetTrace ( ) const
inline

Definition at line 314 of file XrdPfcInfo.hh.

314{return m_trace; }

References m_trace.

◆ GetVersion()

int XrdPfc::Info::GetVersion ( )
inline

Get version.

Definition at line 266 of file XrdPfcInfo.hh.

266{ return m_version; }

References m_version.

◆ HasNoCkSumTime()

bool XrdPfc::Info::HasNoCkSumTime ( ) const
inline

Definition at line 299 of file XrdPfcInfo.hh.

299{ return m_store.m_noCkSumTime != 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by ResetCkSumCache(), and ResetCkSumNet().

+ Here is the caller graph for this function:

◆ IsCkSumAny()

bool XrdPfc::Info::IsCkSumAny ( ) const
inline

Definition at line 291 of file XrdPfcInfo.hh.

291{ return m_store.m_status.f_cksum_check & CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumBoth()

bool XrdPfc::Info::IsCkSumBoth ( ) const
inline

Definition at line 292 of file XrdPfcInfo.hh.

292{ return (m_store.m_status.f_cksum_check & CSChk_Both) == CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumCache()

bool XrdPfc::Info::IsCkSumCache ( ) const
inline

Definition at line 289 of file XrdPfcInfo.hh.

289{ return m_store.m_status.f_cksum_check & CSChk_Cache; }

References XrdPfc::CSChk_Cache, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumCache(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsCkSumNet()

bool XrdPfc::Info::IsCkSumNet ( ) const
inline

Definition at line 290 of file XrdPfcInfo.hh.

290{ return m_store.m_status.f_cksum_check & CSChk_Net; }

References XrdPfc::CSChk_Net, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumNet(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsComplete()

bool XrdPfc::Info::IsComplete ( ) const
inline

Get complete status.

Definition at line 447 of file XrdPfcInfo.hh.

448{
449 return m_complete;
450}

References m_complete.

Referenced by XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Read(), XrdPfc::File::ReadV(), XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ Read()

bool Info::Read ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Read content of cinfo file into this object.

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 294 of file XrdPfcInfo.cc.

295{
296 // Does not need lock, called only in File::Open before File::Run() starts.
297 // XXXX Wait, how about Purge, and LocalFilePath, Stat?
298
299 TraceHeader trace_pfx("Read()", dname, fname);
300
301 FpHelper r(fp, 0, m_trace, m_traceID, trace_pfx);
302
303 if (r.Read(m_version)) return false;
304
306 {
307 if (m_version == 2)
308 {
309 return ReadV2(fp, r.f_off, dname, fname);
310 }
311 else if (m_version == 3)
312 {
313 return ReadV3(fp, r.f_off, dname, fname);
314 }
315 else
316 {
317 TRACE(Warning, trace_pfx << "File version " << m_version << " not supported.");
318 return false;
319 }
320 }
321
322 uint32_t cksum;
323
324 if (r.Read(m_store) || r.Read(cksum)) return false;
325
326 if (cksum != CalcCksumStore())
327 {
328 TRACE(Error, trace_pfx << "Checksum Store mismatch.");
329 return false;
330 }
331
332 ResizeBits();
334
335 if (r.ReadRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
336 r.ReadRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
337 r.Read(cksum))
338 {
339 return false;
340 }
341
342 if (cksum != CalcCksumSyncedAndAStats())
343 {
344 TRACE(Error, trace_pfx << "Checksum Synced or AStats mismatch.");
345 return false;
346 }
347
349
351
352 return true;
353}
#define TRACE(act, x)
Definition XrdTrace.hh:63
void UpdateDownloadCompleteStatus()
Update complete status.
uint32_t CalcCksumStore()
Get cksum, MD5 is for backward compatibility with V2 and V3.
void ResizeBits()
Reserve bit vectors for file_size / buffer_size bytes.
static const int s_defaultVersion
uint32_t CalcCksumSyncedAndAStats()
static const char * m_traceID
int m_astatSize
size of AStat vector
Definition XrdPfcInfo.hh:84

References CalcCksumStore(), CalcCksumSyncedAndAStats(), Error, GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_buff_written, m_store, m_trace, m_traceID, m_version, ResizeBits(), s_defaultVersion, TRACE, and UpdateDownloadCompleteStatus().

Referenced by XrdPfc::Cache::DetermineFullFileSize(), and XrdPfc::Cache::LocalFilePath().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ RefAStats()

const std::vector< AStat > & XrdPfc::Info::RefAStats ( ) const
inline

Definition at line 272 of file XrdPfcInfo.hh.

272{ return m_astats; }

References m_astats.

◆ RefStoredData()

const Store & XrdPfc::Info::RefStoredData ( ) const
inline

Get stored data.

Definition at line 271 of file XrdPfcInfo.hh.

271{ return m_store; }

References m_store.

◆ ResetAllAccessStats()

void Info::ResetAllAccessStats ( )

Reset IO Stats.

Definition at line 359 of file XrdPfcInfo.cc.

360{
363 m_astats.clear();
364}

References XrdPfc::Info::Store::m_accessCnt, m_astats, XrdPfc::Info::Store::m_astatSize, and m_store.

◆ ResetCkSumCache()

void Info::ResetCkSumCache ( )

Definition at line 203 of file XrdPfcInfo.cc.

204{
205 if (IsCkSumCache())
206 {
207 m_store.m_status.f_cksum_check &= ~CSChk_Cache;
208 if ( ! HasNoCkSumTime())
209 m_store.m_noCkSumTime = time(0);
210 }
211}
bool IsCkSumCache() const
bool HasNoCkSumTime() const

References HasNoCkSumTime(), IsCkSumCache(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

+ Here is the call graph for this function:

◆ ResetCkSumNet()

void Info::ResetCkSumNet ( )

Definition at line 213 of file XrdPfcInfo.cc.

214{
215 if (IsCkSumNet())
216 {
217 m_store.m_status.f_cksum_check &= ~CSChk_Net;
218 if ( ! HasNoCkSumTime())
219 m_store.m_noCkSumTime = time(0);
220 }
221}
bool IsCkSumNet() const

References HasNoCkSumTime(), IsCkSumNet(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ResetNoCkSumTime()

void XrdPfc::Info::ResetNoCkSumTime ( )
inline

Definition at line 302 of file XrdPfcInfo.hh.

302{ m_store.m_noCkSumTime = 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ ResizeBits()

void Info::ResizeBits ( )

Reserve bit vectors for file_size / buffer_size bytes.

Definition at line 172 of file XrdPfcInfo.cc.

173{
174 // drop buffer in case of failed/partial reads
175
176 if (m_buff_synced) free(m_buff_synced);
179
181
182 m_buff_written = (unsigned char*) malloc(GetBitvecSizeInBytes());
183 m_buff_synced = (unsigned char*) malloc(GetBitvecSizeInBytes());
186
188 m_complete = false;
189
191 {
192 m_buff_prefetch = (unsigned char*) malloc(GetBitvecSizeInBytes());
194 }
195 else
196 {
197 m_buff_prefetch = 0;
198 }
199}

References GetBitvecSizeInBytes(), m_bitvecSizeInBits, m_buff_prefetch, m_buff_synced, m_buff_written, XrdPfc::Info::Store::m_buffer_size, m_complete, XrdPfc::Info::Store::m_file_size, m_hasPrefetchBuffer, m_missingBlocks, and m_store.

Referenced by Read(), and SetBufferSizeFileSizeAndCreationTime().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetAllBitsSynced()

void Info::SetAllBitsSynced ( )

Mark all blocks as synced to disk.

Definition at line 144 of file XrdPfcInfo.cc.

145{
146 // The following should be:
147 // memset(m_buff_synced, 255, GetBitvecSizeInBytes());
148 // but GCC produces an overzealous 'possible argument transpose warning' and
149 // xrootd build uses warnings->errors escalation.
150 // This workaround can be removed for gcc >= 5.
151 // See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61294
152 const int nb = GetBitvecSizeInBytes();
153 for (int i = 0; i < nb; ++i)
154 m_buff_synced[i] = 255;
155
156 m_complete = true;
157}

References GetBitvecSizeInBytes(), m_buff_synced, and m_complete.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::DataFsSnapshot::write_json_file().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitPrefetch()

void XrdPfc::Info::SetBitPrefetch ( int  i)
inline

Mark block as obtained through prefetch.

Definition at line 365 of file XrdPfcInfo.hh.

366{
367 if (!m_buff_prefetch) return;
368
369 const int cn = i/8;
370 assert(cn < GetBitvecSizeInBytes());
371
372 const int off = i - cn*8;
373 m_buff_prefetch[cn] |= cfiBIT(off);
374}

References GetBitvecSizeInBytes(), and m_buff_prefetch.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitSynced()

void XrdPfc::Info::SetBitSynced ( int  i)
inline

Mark block as synced to disk.

Definition at line 387 of file XrdPfcInfo.hh.

388{
389 const int cn = i/8;
390 assert(cn < GetBitvecSizeInBytes());
391
392 const int off = i - cn*8;
393 m_buff_synced[cn] |= cfiBIT(off);
394}

References GetBitvecSizeInBytes(), and m_buff_synced.

Referenced by XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitWritten()

void XrdPfc::Info::SetBitWritten ( int  i)
inline

Mark block as written to disk.

Definition at line 352 of file XrdPfcInfo.hh.

353{
354 const int cn = i/8;
355 assert(cn < GetBitvecSizeInBytes());
356
357 const int off = i - cn*8;
358
359 m_buff_written[cn] |= cfiBIT(off);
360
361 if (--m_missingBlocks == 0)
362 m_complete = true;
363}

References GetBitvecSizeInBytes(), m_buff_written, m_complete, and m_missingBlocks.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBufferSizeFileSizeAndCreationTime()

void Info::SetBufferSizeFileSizeAndCreationTime ( long long  bs,
long long  fs 
)

Definition at line 161 of file XrdPfcInfo.cc.

162{
163 // Needed only when Info object is created for the first time in File::Open()
165 m_store.m_file_size = fs;
166 ResizeBits();
167 m_store.m_creationTime = time(0);
168}

References XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_file_size, m_store, and ResizeBits().

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::DataFsSnapshot::write_json_file().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetCkSumState()

void XrdPfc::Info::SetCkSumState ( CkSumCheck_e  css)
inline

Definition at line 294 of file XrdPfcInfo.hh.

294{ m_store.m_status.f_cksum_check = css; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ TestBitPrefetch()

bool XrdPfc::Info::TestBitPrefetch ( int  i) const
inline

Test if block at the given index has been prefetched.

Definition at line 376 of file XrdPfcInfo.hh.

377{
378 if (!m_buff_prefetch) return false;
379
380 const int cn = i/8;
381 assert(cn < GetBitvecSizeInBytes());
382
383 const int off = i - cn*8;
384 return (m_buff_prefetch[cn] & cfiBIT(off)) != 0;
385}

References GetBitvecSizeInBytes(), and m_buff_prefetch.

+ Here is the call graph for this function:

◆ TestBitWritten()

bool XrdPfc::Info::TestBitWritten ( int  i) const
inline

Test if block at the given index is written to disk.

Definition at line 343 of file XrdPfcInfo.hh.

344{
345 const int cn = i/8;
346 assert(cn < GetBitvecSizeInBytes());
347
348 const int off = i - cn*8;
349 return (m_buff_written[cn] & cfiBIT(off)) != 0;
350}

References GetBitvecSizeInBytes(), and m_buff_written.

Referenced by CountBlocksNotWrittenInRng(), GetLastDownloadedBlock(), GetNDownloadedBlocks(), and XrdPfc::File::Prefetch().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ UpdateDownloadCompleteStatus()

void XrdPfc::Info::UpdateDownloadCompleteStatus ( )
inline

Update complete status.

Definition at line 463 of file XrdPfcInfo.hh.

464{
467}
int CountBlocksNotWrittenInRng(int firstIdx, int lastIdx) const
Check download status in given block range.

References CountBlocksNotWrittenInRng(), m_bitvecSizeInBits, m_complete, and m_missingBlocks.

Referenced by Read().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ Write()

bool Info::Write ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Write number of blocks and read buffer size

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 266 of file XrdPfcInfo.cc.

267{
268 TraceHeader trace_pfx("Write()", dname, fname);
269
271 m_store.m_astatSize = (int32_t) m_astats.size();
272
273 FpHelper w(fp, 0, m_trace, m_traceID, trace_pfx);
274
275 if (w.Write(s_defaultVersion) ||
276 w.Write(m_store) ||
277 w.Write(CalcCksumStore()) ||
278 w.WriteRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
279 w.WriteRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
280 w.Write(CalcCksumSyncedAndAStats()))
281 {
282 return false;
283 }
284
285 return true;
286}
void CompactifyAccessRecords()
Compactify access records to the configured maximum.

References CalcCksumStore(), CalcCksumSyncedAndAStats(), CompactifyAccessRecords(), GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_store, m_trace, m_traceID, s_defaultVersion, and s_maxNumAccess.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Sync(), and XrdPfc::DataFsSnapshot::write_json_file().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStat()

void Info::WriteIOStat ( Stats s)

Write bytes missed, hits, and disk.

Definition at line 429 of file XrdPfcInfo.cc.

430{
431 m_astats.back().NumIos = s.m_NumIos;
432 m_astats.back().Duration = s.m_Duration;
433 m_astats.back().BytesHit = s.m_BytesHit;
434 m_astats.back().BytesMissed = s.m_BytesMissed;
435 m_astats.back().BytesBypassed = s.m_BytesBypassed;
436}
long long m_BytesMissed
number of bytes served from remote and cached
long long m_BytesBypassed
number of bytes served directly through XrdCl
int m_Duration
total duration of all IOs attached
int m_NumIos
number of IO objects attached during this access
long long m_BytesHit
number of bytes served from disk

References m_astats, XrdPfc::Stats::m_BytesBypassed, XrdPfc::Stats::m_BytesHit, XrdPfc::Stats::m_BytesMissed, XrdPfc::Stats::m_Duration, and XrdPfc::Stats::m_NumIos.

Referenced by XrdPfc::File::Sync(), and WriteIOStatDetach().

+ Here is the caller graph for this function:

◆ WriteIOStatAttach()

void Info::WriteIOStatAttach ( )

Write open time in the last entry of access statistics.

Definition at line 420 of file XrdPfcInfo.cc.

421{
423
424 AStat as;
425 as.AttachTime = time(0);
426 m_astats.push_back(as);
427}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, and m_store.

◆ WriteIOStatDetach()

void Info::WriteIOStatDetach ( Stats s)

Write close time together with bytes missed, hits, and disk.

Definition at line 438 of file XrdPfcInfo.cc.

439{
440 m_astats.back().DetachTime = time(0);
441 WriteIOStat(s);
442}
void WriteIOStat(Stats &s)
Write bytes missed, hits, and disk.

References m_astats, and WriteIOStat().

Referenced by XrdPfc::File::FinalizeSyncBeforeExit().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [1/2]

void Info::WriteIOStatSingle ( long long  bytes_disk)

Write single open/close time for given bytes read from disk.

Definition at line 444 of file XrdPfcInfo.cc.

445{
447
448 AStat as;
449 as.AttachTime = as.DetachTime = time(0);
450 as.NumIos = 1;
451 as.BytesHit = bytes_disk;
452 m_astats.push_back(as);
453}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [2/2]

void Info::WriteIOStatSingle ( long long  bytes_disk,
time_t  att,
time_t  dtc 
)

Write open/close with given time and bytes read from disk.

Definition at line 455 of file XrdPfcInfo.cc.

456{
458
459 AStat as;
460 as.AttachTime = att;
461 as.DetachTime = dtc;
462 as.NumIos = 1;
463 as.Duration = dtc - att;
464 as.BytesHit = bytes_disk;
465 m_astats.push_back(as);
466}

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Member Data Documentation

◆ m_astats

◆ m_bitvecSizeInBits

int XrdPfc::Info::m_bitvecSizeInBits
protected

◆ m_buff_prefetch

unsigned char* XrdPfc::Info::m_buff_prefetch
protected

prefetch statistics

Definition at line 322 of file XrdPfcInfo.hh.

Referenced by ~Info(), ResizeBits(), SetBitPrefetch(), and TestBitPrefetch().

◆ m_buff_synced

unsigned char* XrdPfc::Info::m_buff_synced
protected

disk written state vector

Definition at line 320 of file XrdPfcInfo.hh.

Referenced by ~Info(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitSynced(), and Write().

◆ m_buff_written

unsigned char* XrdPfc::Info::m_buff_written
protected

download state vector

Definition at line 321 of file XrdPfcInfo.hh.

Referenced by ~Info(), Read(), ResizeBits(), SetBitWritten(), and TestBitWritten().

◆ m_complete

bool XrdPfc::Info::m_complete
protected

cached; if false, set to true when missingBlocks hit zero

Definition at line 328 of file XrdPfcInfo.hh.

Referenced by IsComplete(), ResizeBits(), SetAllBitsSynced(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_hasPrefetchBuffer

bool XrdPfc::Info::m_hasPrefetchBuffer
protected

constains current prefetch score

Definition at line 329 of file XrdPfcInfo.hh.

Referenced by ResizeBits().

◆ m_missingBlocks

int XrdPfc::Info::m_missingBlocks
protected

cached, updated in SetBitWritten()

Definition at line 327 of file XrdPfcInfo.hh.

Referenced by ResizeBits(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_store

◆ m_trace

XrdSysTrace* XrdPfc::Info::m_trace
protected

Definition at line 317 of file XrdPfcInfo.hh.

Referenced by GetTrace(), Read(), and Write().

◆ m_traceID

const char * Info::m_traceID = "CInfo"
static

Definition at line 308 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ m_version

int XrdPfc::Info::m_version
protected

Definition at line 325 of file XrdPfcInfo.hh.

Referenced by GetVersion(), and Read().

◆ s_defaultVersion

const int Info::s_defaultVersion = 4
static

Definition at line 312 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ s_infoExtension

◆ s_infoExtensionLen

const size_t Info::s_infoExtensionLen = strlen(Info::s_infoExtension)
static

◆ s_maxNumAccess

size_t Info::s_maxNumAccess = 20
static

Definition at line 311 of file XrdPfcInfo.hh.

Referenced by CompactifyAccessRecords(), XrdPfc::Cache::Config(), and Write().


The documentation for this class was generated from the following files: