Skip to content

[RF][HS3] HS3 v0.3 conformance: follow HS3 #135 and #124 - #23840

Draft
cburgard wants to merge 9 commits into
root-project:masterfrom
cburgard:hs3-conformance-fixes
Draft

cburgard wants to merge 9 commits into
root-project:masterfrom
cburgard:hs3-conformance-fixes

Conversation

@cburgard

@cburgard cburgard commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Important

#23996 is merged; this PR is based on master, and all 9 commits are for review. It stays in draft until HS3 PRs #135 and #124 are merged.

Rebased onto master after the merge of #23996, including the follow-up commits added there during review (e.g. isHS3VersionBefore() instead of isLegacyHS3Version(), analyses named after their ModelConfig).

This Pull request:

Follows up on #23996 (small HS3 conformance fixes against HS3
master at cb3e9eb) with the changes that depend on pending HS3 v0.3 PRs: the normative JSON fixes of HS3
#135, and HistFactory constraints in the form of HS3 #124 and #129
(constraint distributions referenced by name). Larger structural items are left to the PRs
and issues stacked on this one:

  • categorical axes (HS3 #108) and the bin order (HS3 #126): #24001;
  • combined distributions and data sets (HS3 #119): #24005;
  • array_dist (HS3 #109), also for the HistFactory constraint terms: #24013;
  • the histfactory function of #129: #23985, deferred until HS3 #143 is settled;
  • name clashes between different kinds of objects in RooFit's export: #24020.

Background: the HS3–RooFit compatibility audit (October 2026).
Related ROOT PR: #23839 (pinned HS3TestSuite with an
expected-failure list).

The PR has two groups of commits (Group I, the HS3-independent part, is #23996):

  • Group II (5 commits, labelled [after HS3 json-fixes]) follows HS3 PR
    #135 "Small normative JSON fixes for v0.3" (branch json-fixes-0.3).
  • Group III (4 commits, labelled [after HS3 #124/#129] or [after HS3 #124]) follows HS3 PRs #124
    "Histfactory Specfication Sync" and #129 "Move HistFactory to an appendix and add the
    histfactory function" (merged).

If #124 is delayed, Group II can be merged on its own.

Each commit builds on its own and passes testRooFitHS3, testJSONInterface and testHistFactory.

Pre-flight findings

Behaviour with #23996 applied:

  1. HistFactory modifier constraints on export (exportChannel() in JSONFactories_HistFactory.cxx): RooFit
    writes "constraint": "<name of the constraint distribution>" (e.g. "alpha_syst1Constraint") and exports that
    distribution separately; the staterror modifier carries "constraint": "Poisson". On import, constraint is
    first resolved as a distribution name and then as a legacy type (Gauss, ...). Group II keeps this; Group III
    replaces it by the parameters/constraints lists of HS3 #124.
  2. Infinite domain bounds (Domains.cxx, ProductDomain::writeJSON()): RooFit writes infinite bounds as JSON
    null. The axis itself is always written; there is no "skip the axis entirely" shortcut. Group II item 3
    therefore changes null to an omitted key.

Changes or fixes:

Group II: after HS3 PR #135

Checked against HS3 PR #135 at head 3ec2f79 (2026-11-05). Compared with the head
0e5d5a3 originally checked, #135 moved its changes to binned data axes to HS3 PR #126 and issue
#137, split the Crystal Ball into a single-sided and a double-sided type (item 6), and left
shapefactor/shapesys to HS3 PR #124. Re-check this table if the PR changes before merging.

# Item Change
1 Version value Export hs3_version: "0.3".
2 Analysis keys Export already wrote parameters_of_interest and domains (with NP/GO/POI domains as entries). Now also import the legacy parameter_of_interest and domain (a string), with a deprecation warning. Test: AnalysisLegacySingularKeys.
3 Infinite bounds Omit infinite min/max of domain axes instead of writing null, and write a default binning only for bounded axes (the HS3 PR requires both bounds for nbins). Name-only axes are kept (valid per the HS3 PR); RooFit needs them to tell a RooRealVar from a RooConstVar. null is still read.
4 Irregular axes No change: #135 no longer touches binned data axes. RooFit writes min/max next to edges (Group III item 2), which is valid in any case.
5 HistFactory errors The HS3 PR makes errors optional: a sample without errors contributes no statistical uncertainty to the staterror. RooFit used the content as squared error instead (i.e. assumed Poisson uncertainties); that is kept for documents before HS3 0.3. In 0.3 documents this only matters if the staterror is constrained via the legacy constraint key. Test: HistFactorySampleWithoutErrors.
6 Crystal Ball crystalball_dist is the single-sided Crystal Ball (sigma, alpha, n), crystalball_doublesided_dist the double-sided one (alpha_L/n_L/alpha_R/n_R, and sigma or sigma_L/sigma_R, no symmetric shorthand). RooCBShape and single-sided RooCrystalBall → crystalball_dist (read back as RooCBShape; the single-sided RooCrystalBall used to be written as an incomplete crystalball_doublesided_dist that could not be read back). Double-sided RooCrystalBall → crystalball_doublesided_dist, with sigma if both sides share it and the tails always written for both sides. Invalid field combinations are rejected. crystalball_dist with _L/_R fields (the double-sided form before HS3 0.3) is still read as RooCrystalBall, with a deprecation warning. Test: CrystalBallTypes.

Group III: after HS3 PRs #124 and #129

Checked against #124 at head bb3b367 (2026-11-06; originally bc5f670, since then #124 merged HS3 master, added the custom modifier, made the modifier name optional and deprecated shapesys) and #129 at head e76fc4a.

# Item File(s) Change Test
1 Constraints by name JSONFactories_HistFactory.cxx Import parameters and constraints (names of distributions, null = unconstrained) for all modifiers; they take precedence over the legacy parameter, constraint, constraint_name and constraint_type, which are still read. staterror uses the named constraints instead of creating Poisson terms; without any constraint information its gammas are unconstrained for hs3_version ≥ 0.3 (older documents get the implicit Poisson terms as before). Export writes parameters/constraints and exports the constraint distributions; the staterror RooPoisson terms become named poisson_dist. parameter, constraint, shapesys data.vals and the channel-level staterror are no longer written. HistFactoryConstraintsByName, updated HistFactoryStatErrorConstraint, HistFactoryConstraintKeyMigration, HistFactoryLegacyConstraintTypeInConstraintKey, HistFactoryModifierWithoutConstraintIsUnconstrained
2 Axis range RooJSONFactoryWSTool.cxx Binned axes written via exportAxis() always carry min/max, also with edges. #124 requires them on histfactory_dist axes; #135 allows them if they equal the first and last edge. IrregularBinnedAxesHaveRange
3 Tutorial tutorials/roofit/roofit/rf515_hfJSON.json Modifiers use parameters/constraints; the formerly implicit constraints are explicit gaussian_dist (mean 0, sigma 1) and poisson_dist (staterror, tau 400/100) entries; hs3_version 0.3. fitted from C++, see below
4 shapefactor JSONFactories_HistFactory.cxx #124 makes shapesys a deprecated alias of shapefactor, whose parameters are constrained by the distributions in constraints, if given, and free otherwise. HistFactory ShapeSys and ShapeFactor are both written as shapefactor (constraints with null for free gammas, omitted if all are free); the type is no longer derived from the name of the ParamHistFunc, so free gammas no longer make the export fail with "cannot find constraint", and constraints of any type are allowed. shapesys is read as shapefactor, with a deprecation warning (once per channel) for HS3 ≥ 0.3. The ParamHistFunc is named <channel>_<name>_ShapeSys if constrained and <channel>_<name>_shapeFactor if free, as in HistFactory. Without constraints, constraints from data.vals are only created for documents before HS3 0.3 or with a legacy constraint key; otherwise data.vals is ignored with a warning. HistFactoryShapeFactor, updated HistFactory test of the ShapeFactor import

Notes on Group III:

  • histfactory function (#129): not implemented. It is a new primitive; it's left for a follow-up,
    #23985. #129's statement that the constraint terms belong to the prediction is how RooFit already builds
    HistFactory models.
  • staterror constraint type: #124 recommends barlow_beeston_lite_poisson_constraint_dist, which RooFit can't
    read or write. Since #124 allows any distribution type, the existing RooPoisson terms are exported as
    poisson_dist. barlow_beeston_lite_poisson_constraint_dist is going to be deprecated in favour of array_dist
    (HS3 #109), which is then to be recommended for the staterror instead; RooFit's array_dist support, including
    the grouping of HistFactory constraints, is #24013. No importer for the BB-lite distribution is planned.
  • Modifier name: optional in #124 (required for custom modifiers, where it names the function). RooFit
    writes and reads it as before.
  • #124 vs. #135: #124 requires min/max on
    histfactory_dist axes. Do not allow short reads. #135 no longer touches binned axes, so the two don't conflict any more.
  • Item 10c of #23996 (staterror constraint on the modifier)
    is kept there, so that [RF][HS3] Bring the HS3 import and export closer to the HS3 standard #23996 is correct against the current standard and can be merged on its own; Group III
    replaces its export.

Behaviour changes users may notice

In addition to those of #23996:

  • Group II: hs3_version is "0.3", and infinite domain bounds are omitted instead of null.
  • Group II: since RooFit now writes hs3_version "0.3", the duplicate-name check introduced in [RF][HS3] Bring the HS3 import and export closer to the HS3 standard #23996 would make RooFit unable to read its own output: its export still produces clashes between different kinds of objects (a named range, a snapshot or a data set named like a pdf), which HS3 does not allow. From 0.3 on, a name used twice within one section, or by a distribution and a function, is still an error; other clashes only warn, as before 0.3 (test NameClashesInOlderDocuments extended accordingly). Avoiding these clashes on export is left for a follow-up: [RF][HS3] Avoid name clashes between different kinds of objects in the HS3 export #24020.
  • Group II: a double-sided RooCrystalBall is written as crystalball_doublesided_dist (as on master, but now
    also with a single sigma), a single-sided one as crystalball_dist. crystalball_dist with _L/_R fields
    triggers a deprecation warning.
  • Group II: in documents with hs3_version ≥ 0.3, a HistFactory sample without errors no longer contributes
    Poisson uncertainties to the staterror.
  • Group III: HistFactory modifiers are written with parameters/constraints and named constraint
    distributions (including poisson_dist terms for staterror); parameter, constraint, shapesys
    (now shapefactor), its data.vals and the channel-level staterror are gone. Free ShapeFactor gammas can be
    exported. In documents with hs3_version ≥ 0.3, a staterror
    without constraints is unconstrained. Binned axes with edges also carry min/max.
  • rf515_hfJSON: with explicit, named constraints its likelihood equals the one of master up to an additive
    constant of 1.3e-5, checked at several parameter points; the fit agrees within the minimizer tolerance
    (mu 1.10955 vs. 1.10947).

Changed tests and reference data

No reference JSON files exist for these tests; expectations are inline. The following existing expectations or
fixtures were changed because of intended behaviour changes:

  • RooGaussianConstVarSigmaExport, ProductDomainEntriesExportExplicitBounds (Group II.3): expect omitted
    instead of null infinite bounds. The second test now also checks that null is still read.
  • tutorials/roofit/roofit/rf515_hfJSON.json (Group III): rewritten with named constraint distributions.
  • HistFactoryConstraintKeyMigration, HistFactoryLegacyConstraintTypeInConstraintKey,
    HistFactoryModifierWithoutConstraintIsUnconstrained, HistFactoryStatErrorConstraint (Group III): expect the
    exported constraints list instead of constraint.
  • HS3ImportShapeFactorModifier in testHistFactory (Group III.4): round-trips a free ShapeFactor, written as shapefactor
    without constraints; it needed constant gammas before.

HistFactory round trip

roofit/histfactory/test/ref_6.16_example_UsingC_combined_meas_model.root (workspace combined), exported with
#23996 and with this branch. Every changed key:

  • Group II: metadata.hs3_version: "0.2" → "0.3"
  • Group III: on every modifier, parameter → parameters (list) and constraint → constraints (list of
    names; for staterror the names of the gamma constraints instead of "Poisson"); the channel-level
    staterror.constraint_type is gone; new distributions gamma_stat_channel1_bin_{0,1}_constraint
    (poisson_dist), functions gamma_stat_channel1_bin_{0,1}_poisMean (product) and their constant parameters
    gamma_stat_channel1_bin_{0,1}_tau and nom_gamma_stat_channel1_bin_{0,1}.

Checklist:

  • tested changes locally
  • updated the docs (if necessary)

Tests run (Linux, Ubuntu 26.04, GCC 15.2, RelWithDebInfo, -Droofit=ON -Dtesting=ON):
ctest -R 'RooFitHS3|JSONInterface': 105 + 1 tests pass on master; 111 + 1 with this PR (6 new). The two test
binaries and testHistFactory (69 tests) were also built and run for every single commit.

Not run: the PyROOT tests (bindings/pyroot/pythonizations/test/roofit.py, which uses HistFactory JSON via
importJSONElement), the rf515_hfJSON.py tutorial itself (its model was fitted from C++ instead), and the
optional HS3 test suite (test_roofit_hs3testsuite). My build has pyroot=OFF.

AI assistance: Claude Code (claude-opus-5-5) implemented the changes, the tests and this description from a
written plan, under review of the author. The commits carry Assisted-by: claude-code:claude-opus-5-5.

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown

Test Results

    23 files      23 suites   3d 22h 31m 33s ⏱️
 3 896 tests  2 787 ✅   0 💤 1 109 ❌
79 902 runs  78 358 ✅ 435 💤 1 109 ❌

For more details on these failures, see this check.

Results for commit 625d361.

@cburgard cburgard changed the title [RF][HS3] Small HS3 conformance fixes [RF][HS3] HS3 v0.3 conformance: follow HS3 #135 and #124 (depends on #23996) Nov 5, 2026
@cburgard
cburgard force-pushed the hs3-conformance-fixes branch 4 times, most recently from 91bf247 to 74ff7df Compare November 5, 2026 12:55
guitargeek pushed a commit that referenced this pull request Nov 5, 2026
Fix several deviations of RooFit's HS3 layer from the HS3 standard
(hep-statistics-serialization-standard master at cb3e9eb). None of the
changes depend on pending HS3 PRs; the ones that do follow in #23840.
Stricter checks only apply to documents that declare a recent enough
hs3_version, so files written by older ROOT versions remain readable.

Metadata and names:
* hs3_version must be a string. Its MAJOR.MINOR is parsed, and a
  warning is printed if it can't be parsed or is newer than what RooFit
  writes. Importers can query it with
  RooJSONFactoryWSTool::hs3VersionInput().
* Duplicate names of top-level objects are an error from HS3 0.3 on,
  and only a warning for older documents.
* On export, likelihoods are named "<analysis>_likelihood", analyses are
  named after their ModelConfig, and transformed helper functions that
  several pdfs share are written only once.

Import:
* The "default_domain" is optional. Without it, variables get the range
  of the first domain that lists them.
* Parameters that only appear in a parameter point are created. They
  are constant if "const" is true or if no domain lists them. A string
  value sets the label of a category. Index categories of combined
  distributions are now created before the parameter points.
* Observables of unbinned data without a domain range take their range
  from the "min"/"max" of the data axes. Infinite bounds (null) are
  supported.
* "init" of an analysis is restored as the ModelConfig snapshot under
  the same name. A missing parameter point is reported.
* An explicit "extended" flag of mixture_dist is checked against the
  number of coefficients.
* Malformed input is rejected instead of being read out of bounds:
  covariance and correlation matrices must be square, "errors" must have
  as many entries as "contents", and "edges" must contain at least two
  strictly ascending values.
* Errors of zero in binned data stay zero. Before, they could be read
  as squared weight = content.

Export:
* Axes of unbinned data only carry "name", "min" and "max".
* The ModelConfig snapshot is written as "init" of the analysis, which
  no longer has the side effect of loading the snapshot values.
* Ternaries in formulas are written as ifelse(c, a, b), and ifelse() is
  translated back to a ternary on import.

Type keys (the old keys are still imported, with a deprecation
warning):
* ARGUS_dist -> argus_dist
* binsampling -> binsampling_dist. Before, RooFit could not read back
  its own export.
* rate_extended_dist uses the fields "dist" and "rate" instead of "pdf"
  and "norm".

HistFactory:
* "LogNormal" is accepted as a constraint type, alongside "Lognormal".
  Log-normal constraints of gamma parameters are rejected instead of
  silently becoming Gaussian.
* The parameter of a normfactor is named after "parameter". "name" is
  only the label of the modifier.
* normsys/histosys modifiers without constraint are unconstrained from
  HS3 0.2 on, with one warning per channel. Older documents, and objects
  imported with importJSONElement(), keep the implicit unit Gaussian.
  On export, unconstrained modifiers are allowed. A normsys with a
  function as parameter is refused.
* The staterror constraint type is read from the modifier ("constraint",
  or "constraint_type" as written by ROOT 6.38 and 6.40.00), with the
  channel-level setting as fallback. It is written in both places, so
  Gaussian stat errors are no longer lost in a round trip.
* A modifier without interpolation and without "default_interpolation"
  is an error from HS3 0.3 on. Older documents keep code 4, with a
  deprecation warning.
* Fix the export of normsys modifiers recognized from formulas like
  1+0.1*alpha: hi/lo were written as 0.1/-0.1 instead of the absolute
  factors 1.1/0.9, and the constraint was dropped.

Other:
* Warnings of RooJSONFactoryWSTool are logged on the ObjectHandling
  topic. Before, they went to the IO topic, which no default stream
  shows.
* Add ModelConfig::GetSnapshotName().
* Update the RooFitHS3 developer documentation.
* Make the constraints and interpolation of the rf515_hfJSON tutorial
  explicit. The fit result is unchanged.

Files written with this version use the new type keys and ifelse(),
which ROOT 6.40 and older cannot read.

Assisted-by: claude-code:claude-opus-5-5
HS3 PR "json-fixes-0.3" defines `hs3_version` as a `"MAJOR.MINOR"` string
and the version of the revised standard as `"0.3"`. RooFit now writes this
version. Documents with a newer version are imported with a warning.

This commit depends on the HS3 PR and should only be merged together with
or after it.

Assisted-by: claude-code:claude-opus-5-5
HS3 PR "json-fixes-0.3" renames the analysis keys to
`parameters_of_interest` and `domains` (a list of domain names), which is
what RooFit already writes, including one entry each for the nuisance
parameter, global observable and parameter-of-interest domains.

The keys of HS3 v0.2.9, `parameter_of_interest` and `domain` (a single
name), are now also read, with a deprecation warning. The plural keys take
precedence.

Assisted-by: claude-code:claude-opus-5-5
…f writing null

RooFit wrote infinite bounds of product-domain axes as JSON `null`, which
HS3 doesn't allow. HS3 PR "json-fixes-0.3" makes `min` and `max` of domain
axes optional, with an absent bound meaning an unbounded range. Infinite
bounds are now omitted on export. Since that PR also requires both bounds
for a binning, the default binning is only written for bounded axes.

Axes without any bound are still written (with only their `name`), which
the HS3 PR keeps valid. RooFit needs them to tell an unbounded RooRealVar
from a RooConstVar. On import, an absent bound means -inf/+inf, and `null`
is still accepted for files written by older ROOT versions.

The RooGaussianConstVarSigmaExport and ProductDomainEntriesExportExplicitBounds
tests are updated to expect omitted instead of null bounds.

Assisted-by: claude-code:claude-opus-5-5
…ll_dist and crystalball_doublesided_dist

HS3 PR root-project#135 defines `crystalball_dist` as the single-sided Crystal Ball
(`sigma`, `alpha`, `n`, as RooCBShape) and the new
`crystalball_doublesided_dist` as the double-sided one (`alpha_L`, `n_L`,
`alpha_R`, `n_R`, and `sigma` or `sigma_L`/`sigma_R`, without a symmetric
shorthand).

* RooCBShape is exported as `crystalball_dist`, as before.
* A single-sided RooCrystalBall is exported as `crystalball_dist`, and read
  back as a RooCBShape (same function, same sign convention for alpha).
  Before, it was written as an incomplete `crystalball_doublesided_dist`
  without `alpha_R`/`n_R`, which could not be read back.
* A double-sided RooCrystalBall is exported as
  `crystalball_doublesided_dist`, with `sigma` if both sides share it, and
  the tails always as `alpha_L`/`n_L`/`alpha_R`/`n_R`.
* `crystalball_doublesided_dist` is imported as RooCrystalBall, also with a
  single `sigma`, and invalid field combinations are rejected.
* `crystalball_dist` with any `_L`/`_R` field is the double-sided form of
  HS3 versions before 0.3. It is still read as RooCrystalBall (including
  `alpha`/`n` for both tails with `sigma_L`/`sigma_R`), with a one-time
  deprecation warning.

The `crystalball_doublesided_dist` factory expression and the RooCrystalBall
export keys are removed, since the new importer and exporter handle all
cases.

Assisted-by: claude-code:claude-opus-5-5
…on't contribute to the staterror

HS3 PRs root-project#135 and root-project#124 make the "errors" of a HistFactory sample optional: a
sample without "errors" contributes no statistical uncertainty to a
staterror modifier. RooFit used the content as squared error instead, i.e.
assumed Poisson uncertainties. That is kept for documents before HS3 0.3.

In HS3 >= 0.3 documents, this only matters if the staterror is constrained
through the legacy "constraint" key, since otherwise the gammas are either
unconstrained or use the named constraint distributions.

Assisted-by: claude-code:claude-opus-5-5
…eference constraints by name

HS3 PR root-project#124 replaces the `parameter` and `constraint` (a constraint type
like `Gauss`) of HistFactory modifiers with the lists `parameters` and
`constraints`. Each entry of `constraints` names a distribution in the
top-level `distributions`, and `null` leaves a parameter unconstrained.
`shapesys` loses its `data.vals`, since the uncertainties are encoded in
the constraint distributions. HS3 PR root-project#129 states that the constraint terms
belong to the prediction, which is how RooFit already builds the model.

Import:
* `parameters` and `constraints` are read for all modifiers, and take
  precedence over the legacy `parameter`, `constraint`, `constraint_name`
  and `constraint_type`, which are still read for older files.
* For `staterror`, the named constraint distributions are used instead of
  creating Poisson terms. Without any constraint information, the gammas
  are unconstrained in documents with `hs3_version` >= 0.3; older documents
  still get the Poisson constraints that RooFit used to create implicitly.
* The `nom_*` global observable is now only created when RooFit builds a
  constraint itself (legacy `Gauss`).

Export:
* All modifiers write `parameters` and, if any parameter is constrained,
  `constraints` with the names of the constraint distributions, which are
  exported alongside. `parameter`, `constraint` and `shapesys` `data.vals`
  are no longer written.
* The RooPoisson (or RooGaussian) constraint terms of the staterror gammas
  are exported as named `poisson_dist` (or `gaussian_dist`) and referenced
  from the staterror modifiers. root-project#124 recommends a
  `barlow_beeston_lite_poisson_constraint_dist`, which RooFit doesn't
  support yet; any distribution type is allowed.
* The channel-level `staterror.constraint_type` and the `constraint` type
  on staterror modifiers (written since the previous HistFactory commits)
  are no longer written, but still read.

Tests that checked the exported `constraint` key are updated to the new
form, and a new test covers import and export by name.

This commit depends on HS3 PRs root-project#124 and root-project#129 and should only be merged
together with or after them.

Assisted-by: claude-code:claude-opus-5-5
…max on irregular binned axes

RooFit wrote only `edges` for binned axes with an irregular binning. HS3
PR root-project#124 requires `name`, `min`, `max` and exactly one of `nbins` or
`edges` on the axes of a `histfactory_dist`, while HS3 PR root-project#135 makes
`min`/`max` optional next to `edges`, but allows them if they equal the
first and last edge. Writing them satisfies both, so binned axes written
via `exportAxis()` (binned data, histograms, ParamHistFunc and HistFactory
axes) now always carry `min` and `max`. The `axes` of generic formulas are
unchanged, because their reader requires either `edges` or
`min`/`max`/`nbins`.

Assisted-by: claude-code:claude-opus-5-5
… named constraints in rf515_hfJSON

Write the HistFactory modifiers of the rf515_hfJSON tutorial in the form
of HS3 PR root-project#124: every modifier names its parameters in `parameters`, and
its constraint distributions by name in `constraints`. The constraint
terms that RooFit used to create implicitly are now explicit:

* unit Gaussians (mean 0, sigma 1) for the normsys parameters, as
  recommended by root-project#124;
* Poisson terms for the two staterror gammas, with the tau values
  (400 and 100) that follow from the sample errors. root-project#124 recommends a
  `barlow_beeston_lite_poisson_constraint_dist`, which RooFit doesn't
  support yet, so `poisson_dist` is used.

The gammas are declared in a domain and in the default parameter point, so
that the functions referring to them can be imported. The document now
declares `hs3_version` 0.3.

The likelihood is the same as before up to an additive constant of
1.3e-5 (checked at several parameter points), and the fit result agrees
within the minimizer tolerance.

Assisted-by: claude-code:claude-opus-5-5
… shapefactor as "shapefactor"

Following the proposal for HS3 PR root-project#124, HS3 has a single per-bin modifier
type, `shapefactor`, whose parameters are constrained by the distributions
named in `constraints`, if given, and free otherwise. `shapesys` is a
deprecated alias.

* Export: HistFactory's ShapeSys and ShapeFactor are both written as
  `shapefactor`, with the constraint distributions found for the gammas in
  `constraints` (null for free gammas, omitted if all are free). The type is
  no longer derived from the name of the ParamHistFunc, so free gammas no
  longer make the export fail with "cannot find constraint", and constraints
  of any type are allowed (the unused relative errors, which only supported
  Gaussian and Poisson constraints, are no longer computed).
* Import: `shapesys` is read as `shapefactor`, with a deprecation warning
  (once per channel) for HS3 >= 0.3. The ParamHistFunc is named
  `<channel>_<name>_ShapeSys` if constrained and
  `<channel>_<name>_shapeFactor` if free, as in HistFactory.
* Without `constraints`, RooFit only creates constraints from `data.vals`
  for documents before HS3 0.3 or with a legacy constraint key; otherwise the
  parameters are free and `data.vals` is ignored with a warning.

The HistFactory test of the HS3 import of ShapeFactors now round-trips a
free ShapeFactor, which needed constant gammas before.

Assisted-by: claude-code:claude-opus-5-5
@cburgard
cburgard force-pushed the hs3-conformance-fixes branch from 74ff7df to 6115e08 Compare November 6, 2026 06:16
@cburgard cburgard changed the title [RF][HS3] HS3 v0.3 conformance: follow HS3 #135 and #124 (depends on #23996) [RF][HS3] HS3 v0.3 conformance: follow HS3 #135 and #124 Nov 6, 2026
dpiparo pushed a commit that referenced this pull request Nov 6, 2026
Fix several deviations of RooFit's HS3 layer from the HS3 standard
(hep-statistics-serialization-standard master at cb3e9eb). None of the
changes depend on pending HS3 PRs; the ones that do follow in #23840.
Stricter checks only apply to documents that declare a recent enough
hs3_version, so files written by older ROOT versions remain readable.

Metadata and names:
* hs3_version must be a string. Its MAJOR.MINOR is parsed, and a
  warning is printed if it can't be parsed or is newer than what RooFit
  writes. Importers can query it with
  RooJSONFactoryWSTool::hs3VersionInput().
* Duplicate names of top-level objects are an error from HS3 0.3 on,
  and only a warning for older documents.
* On export, likelihoods are named "<analysis>_likelihood", analyses are
  named after their ModelConfig, and transformed helper functions that
  several pdfs share are written only once.

Import:
* The "default_domain" is optional. Without it, variables get the range
  of the first domain that lists them.
* Parameters that only appear in a parameter point are created. They
  are constant if "const" is true or if no domain lists them. A string
  value sets the label of a category. Index categories of combined
  distributions are now created before the parameter points.
* Observables of unbinned data without a domain range take their range
  from the "min"/"max" of the data axes. Infinite bounds (null) are
  supported.
* "init" of an analysis is restored as the ModelConfig snapshot under
  the same name. A missing parameter point is reported.
* An explicit "extended" flag of mixture_dist is checked against the
  number of coefficients.
* Malformed input is rejected instead of being read out of bounds:
  covariance and correlation matrices must be square, "errors" must have
  as many entries as "contents", and "edges" must contain at least two
  strictly ascending values.
* Errors of zero in binned data stay zero. Before, they could be read
  as squared weight = content.

Export:
* Axes of unbinned data only carry "name", "min" and "max".
* The ModelConfig snapshot is written as "init" of the analysis, which
  no longer has the side effect of loading the snapshot values.
* Ternaries in formulas are written as ifelse(c, a, b), and ifelse() is
  translated back to a ternary on import.

Type keys (the old keys are still imported, with a deprecation
warning):
* ARGUS_dist -> argus_dist
* binsampling -> binsampling_dist. Before, RooFit could not read back
  its own export.
* rate_extended_dist uses the fields "dist" and "rate" instead of "pdf"
  and "norm".

HistFactory:
* "LogNormal" is accepted as a constraint type, alongside "Lognormal".
  Log-normal constraints of gamma parameters are rejected instead of
  silently becoming Gaussian.
* The parameter of a normfactor is named after "parameter". "name" is
  only the label of the modifier.
* normsys/histosys modifiers without constraint are unconstrained from
  HS3 0.2 on, with one warning per channel. Older documents, and objects
  imported with importJSONElement(), keep the implicit unit Gaussian.
  On export, unconstrained modifiers are allowed. A normsys with a
  function as parameter is refused.
* The staterror constraint type is read from the modifier ("constraint",
  or "constraint_type" as written by ROOT 6.38 and 6.40.00), with the
  channel-level setting as fallback. It is written in both places, so
  Gaussian stat errors are no longer lost in a round trip.
* A modifier without interpolation and without "default_interpolation"
  is an error from HS3 0.3 on. Older documents keep code 4, with a
  deprecation warning.
* Fix the export of normsys modifiers recognized from formulas like
  1+0.1*alpha: hi/lo were written as 0.1/-0.1 instead of the absolute
  factors 1.1/0.9, and the constraint was dropped.

Other:
* Warnings of RooJSONFactoryWSTool are logged on the ObjectHandling
  topic. Before, they went to the IO topic, which no default stream
  shows.
* Add ModelConfig::GetSnapshotName().
* Update the RooFitHS3 developer documentation.
* Make the constraints and interpolation of the rf515_hfJSON tutorial
  explicit. The fit result is unchanged.

Files written with this version use the new type keys and ifelse(),
which ROOT 6.40 and older cannot read.

Assisted-by: claude-code:claude-opus-5-5
(cherry picked from commit d61fecf)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants