Skip to content

MModuleEventSaver: with split files, events before the first split time are lost, and events with only a UTC time (e.g. from HDF5) are all lost #263

Open

Description

@zoglauer

This issue was written by Claude under the direction of @zoglauer.

The unit test UTNModuleEventSaver (PR #247) marks both problems below as known failures with ExpectedToFail() (5 checks). All links refer to src/MModuleEventSaver.cxx unless noted.

Splitting the output by time is enabled by default: the constructor sets m_SplitFile = true (L92-L93), so a newly added event saver has "Split the file" checked in the GUI. With splitting, AnalyzeEvent() decides on the sub-file like this:

MFile* Choosen = 0; // Wish C++ would allow unassigned references...
if (m_SplitFile == true) {
MTime Current = Event->GetTime();
if (Current > m_SubFileStart + m_SplitFileTime) {
m_SubFileStart = Current;
if (StartSubFile() == false) {
m_IsOK = false;
return false;
}
}
Choosen = &m_SubFileOut;
} else {
Choosen = &m_Out;

A new sub-file is only started when the event time is later than m_SubFileStart + m_SplitFileTime, and m_SubFileStart starts at 0 (L124). Until then the event is written to m_SubFileOut, which is not open yet, so it is silently lost.

1. Events before the first split time are lost

With a split time of 10 s, an event at 5 s does not start a sub-file (5 s is not later than 0 + 10 s) and is lost; only the events from 100 s on are written. With the default split time of 600 s, every event in the first 10 minutes after the epoch of the event time is lost.

2. Events with only a UTC time are all lost

The split uses GetTime() (L486), the generic time of MReadOutSequence. The HDF5 loader only sets the UTC time (MModuleLoaderMeasurementsHDF.cxx L913-L919), so GetTime() is 0 for all events. No sub-file is ever started, and with split files enabled all events are lost.

All committed configurations (the end-to-end and loader test configurations and the examples in resource/examples) set <SplitFile>false</SplitFile>, so none of them exercises the splitting.

Suggested fix

Either:

  • Fix the splitting: start the first sub-file with the first event (e.g. if no sub-file is open yet), and split by the time the loaders actually set (e.g. the UTC time, falling back to GetTime()); use the same time for the sub-file names (L287-L335).
  • Or remove the time splitting, if it is no longer used: the option, its GUI elements and XML keys, and the related checks in the unit test. In any case, it should not be enabled by default while it loses data.

Done when: all problems described in this issue are fixed, all ExpectedToFail() instances related to this issue are removed from the unit tests, and the unit tests pass on both macOS and Linux.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions