Skip to content

Commit 42ff11a

Browse files
committed
CCDB: read std::array objects written by libstdc++ on libc++ builds
Needs review. The only member of std::array<T,N> is T _M_elems[N] in libstdc++ and T __elems_[N] in libc++, so the StreamerInfo of an array written on Linux has a checksum (0x9189320d for array<int,540>) that the macOS class does not have (0x919bc501). When the file is opened, TStreamerInfo::BuildCheck (ROOT 6.40.04, "perfect alias" block, since 2014) drops the on-file StreamerInfo as an alias of the in-memory one, and the read then fails: Error in <TBufferFile::ReadVersion>: Could not find the StreamerInfo with a checksum of 0x9189320d for the class "array<int,540>" in name. Error in <TBufferFile::CheckByteCount>: object of class array<int,540> read too few bytes: 6 instead of 2166 leaving the array zero-filled. TRD/Calib/DCSDPsFedChamberStatus is such an object: every chamber looks off, the TRD digitizer writes 0 digits, and on macOS the full system test has no TRD and no CTF. If the in-memory StreamerInfo already exists when the file is opened, ROOT registers the on-file one but cannot map _M_elems, and reads zeros without any error. Before reading an array<...> object, DataRefUtils::registerLibstdcxxStdArray registers the on-file layout as a class version of its own, with its member renamed to the in-memory name, so ROOT finds it by checksum and reads the T[N] payload, identical in both libraries, as usual. It is called from both copies of extractFromTFile: CcdbApi (CcdbApi/BasicCCDBManager, DPL CCDB fetcher) and DataRefUtils (DPL condition inputs). Everything is inside #if defined(_LIBCPP_VERSION): libstdc++ builds (EPN, Grid, CI) compile exactly the same code as before. Verified on macOS 27.2, ROOT 6.40.04-alice: - BasicCCDBManager read of DCSDPsFedChamberStatus at 1550600800000: 540 zeros before, 521x3 + 19x0 after, the same as decoding the payload by hand; at 1720000000000: 431x3, 89x4, 2x5, 18x0. - array<string,540> (TRD/Calib/DCSDPsFedCFGtag): 540 empty strings before, the real tags after. - TRD-only o2-sim-digitizer-workflow on the 10-event PbPb FST sample: 0 digits before, 126921 after. Plain-ROOT reproducer for the ROOT team: ~/src/data/ccdb-stdarray-libcxx-repro.
1 parent 92368c4 commit 42ff11a

3 files changed

Lines changed: 53 additions & 0 deletions

File tree

‎CCDB/src/CcdbApi.cxx‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,9 @@
5151
#include <string_view>
5252
#include <utility>
5353
#include <TAlienUserAgent.h>
54+
#if defined(_LIBCPP_VERSION)
55+
#include "Framework/DataRefUtils.h"
56+
#endif
5457
#include <unordered_set>
5558
#include "rapidjson/document.h"
5659
#include "rapidjson/writer.h"
@@ -986,6 +989,9 @@ void* CcdbApi::extractFromTFile(TFile& file, TClass const* cl, const char* what)
986989
if (!cl) {
987990
return nullptr;
988991
}
992+
#if defined(_LIBCPP_VERSION)
993+
o2::framework::DataRefUtils::registerLibstdcxxStdArray(file, cl);
994+
#endif
989995
auto object = file.GetObjectChecked(what, cl);
990996
if (!object) {
991997
// it could be that object was stored with previous convention

‎Framework/Core/include/Framework/DataRefUtils.h‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,10 @@ namespace o2::conf
3131
{
3232
class ConfigurableParam;
3333
}
34+
#if defined(_LIBCPP_VERSION)
35+
class TClass;
36+
class TFile;
37+
#endif
3438

3539
namespace o2::framework
3640
{
@@ -185,6 +189,10 @@ struct DataRefUtils {
185189
// Decode a CCDB object using the CcdbApi.
186190
static void* decodeCCDB(DataRef const& ref, std::type_info const& info);
187191
static std::map<std::string, std::string> extractCCDBHeaders(DataRef const& ref);
192+
#if defined(_LIBCPP_VERSION)
193+
/// Make a std::array<T,N> written with libstdc++ readable from file by this libc++ build.
194+
static void registerLibstdcxxStdArray(TFile& file, TClass const* cl);
195+
#endif
188196
/// Return a span over the raw CCDB payload bytes, stripping the flattened HTTP
189197
/// headers footer that CCDBFetcherHelper appends. Use this when the CCDB entry is
190198
/// a binary blob rather than a ROOT-serialised object.

‎Framework/Core/src/DataRefUtils.cxx‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,11 @@
2020
#include <TH1.h>
2121
#include <TError.h>
2222
#include <algorithm>
23+
#if defined(_LIBCPP_VERSION)
24+
#include <TList.h>
25+
#include <TStreamerElement.h>
26+
#include <TStreamerInfo.h>
27+
#endif
2328

2429
namespace o2::framework
2530
{
@@ -31,6 +36,9 @@ void* extractFromTFile(TFile& file, TClass const* cl, const char* what)
3136
if (!cl) {
3237
return nullptr;
3338
}
39+
#if defined(_LIBCPP_VERSION)
40+
DataRefUtils::registerLibstdcxxStdArray(file, cl);
41+
#endif
3442
auto object = file.GetObjectChecked(what, cl);
3543
if (!object) {
3644
// it could be that object was stored with previous convention
@@ -73,6 +81,37 @@ void* extractFromTFile(TFile& file, TClass const* cl, const char* what)
7381
return result;
7482
}
7583
} // namespace
84+
85+
#if defined(_LIBCPP_VERSION)
86+
// The only member of std::array<T,N> is T _M_elems[N] in libstdc++, T __elems_[N] in libc++, so
87+
// objects written on Linux carry a StreamerInfo with a checksum the libc++ class does not have.
88+
// When opening the file, ROOT drops that StreamerInfo as an alias of the in-memory one
89+
// (TStreamerInfo::BuildCheck), and the read fails with "Could not find the StreamerInfo with a
90+
// checksum", leaving the array zero-filled. Register the on-file layout as a version of its own,
91+
// under the in-memory member name, so that the payload, T[N] for both, is read as usual.
92+
void DataRefUtils::registerLibstdcxxStdArray(TFile& file, TClass const* cl)
93+
{
94+
auto* infos = std::strncmp(cl->GetName(), "array<", 6) == 0 ? file.GetStreamerInfoCache() : nullptr;
95+
auto* onfile = infos ? dynamic_cast<TStreamerInfo*>(infos->FindObject(cl->GetName())) : nullptr;
96+
if (!onfile || onfile->GetNelement() != 1 || std::strcmp(onfile->GetElement(0)->GetName(), "_M_elems") != 0) {
97+
return;
98+
}
99+
// Exists before BuildCheck, so that the on-file layout gets a version of its own
100+
auto* current = static_cast<TStreamerInfo*>(const_cast<TClass*>(cl)->GetStreamerInfo());
101+
auto* info = static_cast<TStreamerInfo*>(cl->FindStreamerInfo(onfile->GetCheckSum()));
102+
if (!info) {
103+
info = static_cast<TStreamerInfo*>(onfile->Clone());
104+
info->BuildCheck(&file);
105+
}
106+
// Also when ROOT registered it itself: it then reads nothing, without any error
107+
if (std::strcmp(info->GetElement(0)->GetName(), current->GetElement(0)->GetName()) != 0) {
108+
info->GetElement(0)->SetName(current->GetElement(0)->GetName());
109+
info->Clear("build");
110+
info->BuildOld();
111+
}
112+
}
113+
#endif
114+
76115
// Adapted from CcdbApi private method interpretAsTMemFileAndExtract
77116
// If the former is moved to public, throws on error and could be changed to
78117
// not require a mutex we could use it.

0 commit comments

Comments
 (0)