Skip to content

Handling of stat:sum values in MCPL files - #62

Merged
tkittel merged 8 commits into
mctools:mainfrom
tkittel:mcpl-statsums
Nov 7, 2026
Merged

tkittel merged 8 commits into
mctools:mainfrom
tkittel:mcpl-statsums

Conversation

@tkittel

@tkittel tkittel commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Lots of back and forth here, the issues raised in #61 are not something with a general fits-all solution.

Claude says:

Handling of stat:sum values in MCPL files (fixes #61):

  • MCPLGen divides the weights by stat:sum:initial_ray_count if the input file has it. Files written by the McStas MCPL_output component with weight_mode=1 (the default since McStas 3.10) have this value, and store the weights multiplied by the number of rays. The value of the whole file is always used, also with skip_events or a limited number of events, so the results of jobs which each process a part of a file can be added up. Files without the value are used as before. Files with a value which is unavailable (-1) or not positive give an error. The new parameter normalise_by_initial_ray_count (default true) can be disabled to always use the weights as stored.
  • MCPLWriter writes stat:sum:geant4_nevts (the number of simulated events) and stat:sum:initial_geant4_weights (the sum of the weights of their primary particles, in the same convention as the weights in the file) to its output files. A header comment in the output files explains them.
  • The initial_ray_count convention assumes that each particle is an independent ray, which is usually not the case for files written by Geant4. So by default, MCPLWriter writes the weights as used in Geant4, and records the initial_ray_count of an MCPLGen input file as stat:sum:original_initial_ray_count (which is passed on again when such a file is used as input). With the new opt_initial_ray_count keyword of --mcpl (or setWriteInitialRayCount(True)), it instead records it as stat:sum:initial_ray_count, with the weights in the same convention as in the input file.
  • Custom stat:sum values can be added to all output files, from Python with G4MCPLPlugins.MCPLWriter.addStatSum(key,value), or from C++ via the new G4MCPL/OutputStatSums.hh registry (which MCPLGen also uses to provide the values of its input file). When multiprocessing, values describing the whole job are divided between the processes, so the merged file gets the value.
  • sb_mcplextra_filterfile keeps stat:sum values when filtering particles, but marks them as unavailable when only a part of the file is used due to the -l limit.
  • The ExprParser exception classes now have out-of-line destructors, so exceptions thrown in one module can be caught in another on platforms comparing type information by address (Intel macOS).
  • CI: added an Intel macOS (macos-15-intel) job to condatest.
  • MCPL 2.2.8 or later is now required.

A new test (G4MCPLTests/testmcplstatsums) covers normalisation, partial reading, multiprocessing, unavailable and invalid values, the opt_initial_ray_count keyword, reading output files again, custom values, and sb_mcplextra_filterfile. The reference logs of testmcplgen and testmcplwrite are updated. The documentation of the MCPL support is updated.

assisted-by: claude

…o MCPLWriter output

- MCPLGen divides the weights by stat:sum:initial_ray_count if the input file
  has it (as files from the McStas MCPL_output component with weight_mode=1,
  the default since McStas 3.10, do). The value of the whole file is always
  used, so results of jobs processing parts of a file can be added up. Files
  without the value are used as before, and files with a value which is not
  finalised or not positive give an error. The new parameter
  normalise_by_initial_ray_count (default true) can be disabled to always use
  the weights as stored.
- MCPLWriter writes stat:sum:geant4_nevts (the number of simulated events) and
  stat:sum:initial_geant4_weights (the sum of the weights of their primary
  particles) to the output files. These are collected via a post-generation
  callback on the generator, and are added up when files are merged.
- Require MCPL 2.2.8 or later.

Assisted-by: Claude
… a file

sb_mcplextra_filterfile keeps stat:sum values when filtering particles, but
when only a part of the input file is used due to the -l limit, the values are
marked as unavailable (the file was split rather than filtered). MCPLGen
refuses to normalise by such values by default.

Assisted-by: Claude
If the input file of MCPLGen has a stat:sum:initial_ray_count value, it is
transferred to the output files of MCPLWriter, with the weights written in the
same convention as in the input file (i.e. multiplied by the value when MCPLGen
divided the weights by it), so reading the output again gives the same weights
in Geant4. The value is marked as unavailable unless all particles of the input
file were used. When multiprocessing, the value is only set in the file of the
parent process (the files of the child processes get 0), since the values are
added up when the files are merged.

The information is provided by MCPLGen via the new G4MCPL/InitialRayCountInfo.hh
interface.

Assisted-by: Claude
Do not mark the new methods as override, since the other methods of the class
are not (clang otherwise gives an error with -Werror). Also flush the output of
the messages about the weights, so they are not lost when multiprocessing.

Assisted-by: Claude
The exception classes of ExprParser were header-only, so each module using
them had its own copy of their type information. On platforms where libc++
compares type information by address (Intel macOS), exceptions thrown in one
module (e.g. for invalid filter expressions) could then not be caught by type
in another (e.g. a Python module loaded with RTLD_LOCAL). Defining their
destructors in Exception.cc places the type information in the ExprParser
library only.

Also make testmcplstatsums robust against messages from multiple processes
being interleaved in the output.

Assisted-by: Claude
On Intel macOS, libc++ compares type information by address, which makes it
sensitive to issues with polymorphic classes or exceptions shared between
Python modules, which are not visible on Linux or arm64 macOS.

Assisted-by: Claude
Replace the G4MCPL/InitialRayCountInfo.hh interface (which MCPLWriter found via
dynamic_cast on the generator, which failed on Intel macOS since the
header-only class had separate type information in each Python module) with a
global registry in G4MCPL/OutputStatSums.hh. Components or user code can add
stat:sum entries for all files written by MCPLWriter (with a mode which
determines how the values add up when the files of multiple processes are
merged), and set a factor for the weights written.

MCPLGen uses it to transfer stat:sum:initial_ray_count of its input file. The
registration happens when its parameters are validated, before the simulation
starts and before any fork when multiprocessing. Python code can add constant
values with G4MCPLPlugins.MCPLWriter.addStatSum(key,value).

Assisted-by: Claude
The stat:sum:initial_ray_count convention of instrument simulations assumes
that each particle in a file is an independent ray, which is usually not the
case for files written by Geant4. MCPLWriter therefore no longer transfers the
initial_ray_count of an MCPLGen input file automatically:

- By default, the output files have the weights as used in Geant4, and the
  value of the input file is recorded as stat:sum:original_initial_ray_count.
- With the new opt_irc keyword of --mcpl (or setWriteInitialRayCount(True)),
  the value is recorded as stat:sum:initial_ray_count, with the weights in the
  same convention as in the input file (an error if the input file has no
  valid value).
- stat:sum:initial_geant4_weights is in the same convention as the weights in
  the file.
- Values describing the whole job (the above, and custom values added with
  addStatSum) are divided between the processes when multiprocessing, so the
  merged file gets the value, rather than being written by the parent process
  only. The check of whether all particles of the input file were used is
  removed.

Assisted-by: Claude
@tkittel tkittel changed the title MCPL: normalise MCPLGen weights by initial_ray_count, and add stat:sum values to MCPLWriter output Handling of stat:sum values in MCPL files Nov 7, 2026
@tkittel
tkittel merged commit 6717e38 into mctools:main Nov 7, 2026
23 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCPLGen: normalise the weights by stat:sum:initial_ray_count when the file has it (McStas MCPL_output weight_mode=1)

1 participant