Skip to content

STL collection object in TFile not compatible across OSX and Linux #17216

New issue
New issue
Open
Open
STL collection object in TFile not compatible across OSX and Linux#17216
Assignees
pcanaldpiparo
Labels
bugin:I/O

Description

@ktf
ktf
opened on Dec 6, 2024

Check duplicate issues.

  • Checked for duplicates

Description

As reported this summer in https://root-forum.cern.ch/t/stl-collection-object-in-tfile-not-compatible-across-osx-and-linux/60321

Using:

#include <array>
#include <iostream>
#include <vector>

#include "TFile.h"

#pragma link C++ class std::vector<std::array<ULong64_t, 2>>+ ;

void writeVector() {
  std::vector < std::array < ULong64_t, 2>> test { { 1ull, 2ull }, { 3ull, 4ull }, { 5ull, 6ull } };
  TFile f("test.root", "RECREATE");
  f.WriteObject(&test, "test");
  f.Close();
}

void readVector() {
  TFile f("test.root");
  auto test = f.Get<std::vector<std::array < ULong64_t, 2> > >("test");

  for (auto &a: *test) {
    std::cout << a[0] << " " << a[1] << std::endl;
  }
  f.Close();
}

to write on one platform and then read the result back on the other results in:

Error in <TBufferFile::ReadVersion>: Could not find the StreamerInfo with a checksum of 0x44178e08 for the class "array<ULong64_t,2>" in test.root.
Error in <TBufferFile::CheckByteCount>: object of class array<ULong64_t,2> read too few bytes: 6 instead of 22
Error in <TBufferFile::ReadVersion>: Could not find the StreamerInfo with a checksum of 0x44178e08 for the class "array<ULong64_t,2>" in test.root.
Error in <TBufferFile::CheckByteCount>: object of class array<ULong64_t,2> read too few bytes: 6 instead of 22
Error in <TBufferFile::ReadVersion>: Could not find the StreamerInfo with a checksum of 0x44178e08 for the class "array<ULong64_t,2>" in test.root.
Error in <TBufferFile::CheckByteCount>: object of class array<ULong64_t,2> read too few bytes: 6 instead of 22

Similarly, if I adapt the above to be an std::unordered_map rather than a vector of pairs it crashes with:

[/usr/lib/system/libsystem_platform.dylib] _sigtramp (no debug info)
[/Users/ktf/src/sw/BUILD/44309b19dcd447bff3ead99ccd49cb399f7edcaa/O2/compiled_macros/Users/ktf/src/sw/BUILD/44309b19dcd447bff3ead99ccd49cb399f7edcaa/O2/foo_C.so] readVector() (no
debug info)
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] cling::IncrementalExecutor::executeWrapper(llvm::StringRef, cling::Value*) const (no debug info)
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] cling::Interpreter::RunFunction(clang::FunctionDecl const*, cling::Value*) (no debug info)
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] cling::Interpreter::EvaluateInternal(std::__1::basic_string<char, std::__1::char_traits<char>, std::__
1::allocator<char>> const&, cling::CompilationOptions, cling::Value*, cling::Transaction**, unsigned long) (no debug info)
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] cling::Interpreter::process(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::alloca
tor<char>> const&, cling::Value*, cling::Transaction**, bool) (no debug info)
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] cling::MetaProcessor::process(llvm::StringRef, cling::Interpreter::CompilationResult&, cling::Value*,
bool) (no debug info)
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] HandleInterpreterException(cling::MetaProcessor*, char const*, cling::Interpreter::CompilationResult&,
 cling::Value*) /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/metacling/src/TCling.cxx:2447
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCling.6.32.06.so] TCling::ProcessLine(char const*, TInterpreter::EErrorCode*) /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/
0/core/metacling/src/TCling.cxx:0
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libRint.6.32.06.so] TRint::ProcessLineNr(char const*, char const*, int*) /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/r
int/src/TRint.cxx:0
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libRint.6.32.06.so] TRint::HandleTermInput() /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/rint/src/TRint.cxx:649
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCore.6.32.06.so] TUnixSystem::CheckDescriptors() /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/unix/src/TUnixSystem.c
xx:0
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCore.6.32.06.so] TMacOSXSystem::DispatchOneEvent(bool) /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/macosx/src/TMacO
SXSystem.mm:378
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCore.6.32.06.so] TSystem::InnerLoop() /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/base/src/TSystem.cxx:404
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCore.6.32.06.so] TSystem::Run() /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/base/src/TSystem.cxx:354
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libCore.6.32.06.so] TApplication::Run(bool) /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/base/src/TApplication.cxx:1887
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/lib/libRint.6.32.06.so] TRint::Run(bool) /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/core/rint/src/TRint.cxx:0
[/Users/ktf/src/sw/osx_arm64/ROOT/10b8d555b9-local1/bin/root.exe] main /Users/ktf/src/sw/SOURCES/ROOT/10b8d555b9/0/main/src/rmain.cxx:86

Running on the same platform works as expected.

Reproducer

See description

ROOT version

6.32.6 built with alibuild

Installation method

alibuild

Operating system

macOS + linux (lxplus)

Additional context

This is preventing us to read some calibration objects which were created during datataking and use them to perform analysis tasks on macOS.

Activity

  1. ktf
    added
    bug
    on Dec 6, 2024

  2. pcanal commented on Dec 6, 2024

    @pcanal
    pcanal
    on Dec 6, 2024
    Member

    Does it happens to anything beside a STL collection of std::array? This combination is not yet supported (and sadly not yet explicitly rejected either :( ). [ the recommendation it to use std::collection< struct_wrapping_std_array<..>> ]

  3. pcanal
    assigned
    pcanal
    and
    dpiparo
    on Dec 6, 2024

  4. ktf commented on Dec 6, 2024

    @ktf
    ktf
    on Dec 6, 2024
    ContributorAuthor

    Yes, with unordered_map<uin64_t, double > , which is how we (re)discovered this today.

  5. pcanal commented on Dec 6, 2024

    @pcanal
    pcanal
    on Dec 6, 2024
    Member

    I would not expect the same error message (type) with that. It is likely that uin64_t has a different underlying type on Linux and Macos and hence result in effectively different class name for unordered_map<uin64_t, double >.
    What is the actual error message in this case?

  6. ktf commented on Dec 6, 2024

    @ktf
    ktf
    on Dec 6, 2024
    ContributorAuthor
    root [0]
Attaching file 6382c81b-b32b-11ef-81d0-0aa202c71b9a as _file0...
Warning in <TStreamerInfo::BuildCheck>:
   The StreamerInfo of class o2::grp::GRPEnvVariables read from file 6382c81b-b32b-11ef-81d0-0aa202c71b9a
   has the same version (=1) as the active class but a different checksum.
   You should update the version to ClassDef(o2::grp::GRPEnvVariables,2).
   Do not try to write objects with the current class definition,
   the files will not be readable.

Warning in <TStreamerInfo::CompareContent>: The following data member of
the on-file layout version 1 of class 'o2::grp::GRPEnvVariables' differs from
the in-memory layout version 1:
   unordered_map<string,vector<pair<unsigned long,double> > > mEnvVars; //
vs
   unordered_map<string,vector<pair<ULong64_t,double> > > mEnvVars; //
(TFile *) 0x12db0c0d0
root [1] .q

    is the original error in the production code.

  7. pcanal commented on Dec 6, 2024

    @pcanal
    pcanal
    on Dec 6, 2024
    Member

    Alright. That error is what I expect. If you are using std::uin64_t with the current ROOT code, the schema will be different on different platforms. Until we had support for this typedefs (in the plan for work for 2025), your best solution is to use the ROOT typedef instead (i.e. ULong64_t) or (for this case) unsigned long long would also work. [Two other solutions might be to remove the class version from that class and just rely on checksum or to make the class version platform dependent]

  8. ktf commented on Dec 7, 2024

    @ktf
    ktf
    on Dec 7, 2024
    ContributorAuthor

    If we substitute with unsigned long long, would that require bumping the version to read it correctly?

  9. pcanal commented on Dec 7, 2024

    @pcanal
    pcanal
    on Dec 7, 2024
    Member

    Yes, the only way to avoid the warnings in most cases is to change the type and increase the version number. (You might still get the warning when trying to read in the same process files written on both mac and linux - because 2 conflicting version 1 would be read)

  10. ferdymercury commented on Jul 22, 2025

    @ferdymercury
    ferdymercury
    on Jul 22, 2025
    Collaborator

    Merging from:

    Reading a std::array<Long64_t, 1> from a TFile created on linux does not work on macOS

    #9001

    To reproduce, download http://ccdb-test.cern.ch:8080/download/d64bd450-1619-11ec-a9d5-200114580202 and open it in TBrowser. Works on ubuntu, but on mac I get:

    root [2] Error in <TBufferFile::ReadVersion>: Could not find the StreamerInfo with a checksum of 0xdc6299b4 for the class "array<Long64_t,1>" in /Users/ktf/Downloads/array<Long64_t,1>_1631706076679.root.
Error in <TBufferFile::CheckByteCount>: object of class array<Long64_t,1> read too few bytes: 6 instead of 14

    when clicking on ccdb_object;1.

    Both ROOT 6.24.02 and 6.24.06 seem to be affected.

    Notice that using a std::vector in place of a std::array works as expected.

    PC: Does the reading work properly on the same platform (both no warning and the correct value)? What is the array value for the example provided?

    yes. Reading it back on linux works just fine.
    The actual value should be some timestamp around 1630659771715.
    PC: :) ok. That's what I get, that looked like a random stream :) .. well not quite the same but similar (1631706076551146), I assume it is a different file
    yeah, I cannot check the actual value right now, but it looks correct.

  11. ferdymercury
    marked Reading a std::array<Long64_t, 1> from a TFile created on linux does not work on macOS #9001 as a duplicate of this issue on Jul 22, 2025
  12. ferdymercury
    added
    in:I/O
    on Oct 20, 2025

  13. ktf commented on Nov 7, 2026

    @ktf
    ktf
    on Nov 7, 2026
    ContributorAuthor

    I have been hit by this once more.

    In the end the solution we will probably adopt is:

    AliceO2Group/AliceO2#16072

    which seems to bring back the correct results also on macOS. Can something similar be done at the ROOT level?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugin:I/O

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions