[cburgard]

Important

Depends on #24001 (which depends on #23840). This branch is stacked on it: please review only the last 10 commits. It stays in draft until HS3 PR #119 is merged, and will be rebased when the PRs below it are merged.

Rebased onto master after the merge of #23996, including the follow-up commits added there during review (e.g. isHS3VersionBefore() instead of isLegacyHS3Version(), analyses named after their ModelConfig).

This Pull request:

Combined distributions and data sets in RooFit's HS3 import/export, following HS3 PR #119 ("combined data sets and combined/conditional distributions"):

• a RooSimultaneous is written as a combined_dist and a data set with category observables as a combined data set, instead of the ROOT-internal description in misc.ROOT_internal that only RooFit could read;
• category_conditional_dist (a distribution selected by the value of a categorical parameter) is read and written;
• the likelihood of a ModelConfig pairs the combined objects, and the import checks that pairing.

Files written by earlier ROOT versions, with misc.ROOT_internal.combined_distributions / combined_datasets, are still read.

Changes or fixes:

1. RooSimultaneous ↔ combined_dist

Direction	Change
Export	A RooSimultaneous is a combined_dist. Its columns are the index category or, for a RooSuperCategory index, the input categories, sorted by name. The columns are declared as categorical axes of default_domain (with the states in their order). components has one entry per state with a pdf, with the key as one label per column. A RooSuperCategory index used to be an export error.
Import	A combined_dist is a RooSimultaneous over the column category, or over a RooSuperCategory of the columns in the order of the document. The columns must be declared by categorical axes of a domain; keys are validated against them (length, labels, duplicates). The ROOT-internal form is still read.

The components are written in the order of the columns and their states, which does not depend on the order of the inputs of a RooSuperCategory, so the document is stable under round trips. Category indices are not part of HS3: after a round trip, the index of a state is its position in the domain's categories.

Tests: CombinedDist (structure, round trip, legacy form), CombinedDistSeveralColumns, CombinedDistImportErrors. CombinedObjectsKeepNamesWithoutMisc checks that a document without misc imports with the same names for the pdf, its index category and the data set (they used to be encoded in misc.ROOT_internal), and that the ModelConfig finds its pdf through the likelihood.

2. Combined data sets ↔ combined

Direction	Change
Export	A data set whose observables include RooCategorys is a combined entry with the categories as columns (sorted by name) and one sub-data set per key, named <data>_<label>[_<label>…], written like any other data set. A key is written also without entries: RooFit treats a channel without entries as present and empty. Several category observables used to be an export error.
Export	If the workspace has a RooSimultaneous with the same columns, each sub-data set carries only the observables of its component (as before), and keys without entries for which it has no component are left out. Otherwise all observables are written.
Export	The likelihood of a ModelConfig pairs the RooSimultaneous with the combined data set ("distributions": ["simPdf"], "data": ["obsData"]) instead of listing the components of both. An analysis is written for every combined data set with the same columns.
Import	A combined data set is a RooDataSet with one category column per entry of columns, filled from the components, which are then not imported on their own. One code path for one and for several columns. The components of combined data sets are assembled before the analyses are read, so the ModelConfig observables include the index category, as in the original HistFactory workspace. The ROOT-internal form is still read, and handled after the analyses as before, because its likelihoods list the components.

Tests: CombinedData (structure, round trip, data without pdf, legacy form), CombinedDataSeveralColumns, CombinedDataImportErrors, and HS3RoundTripOfReferenceWorkspace in testHistFactory for the ROOT 6.16 reference workspace (same objects, the same likelihood up to a constant, stable document). The existing SimultaneousFit and HS3ClosureLoop tests now exercise the new form.

3. Pairing checks

HS3 #119 treats a key of a combined_dist without a sub-data set in the paired combined data set as an unobserved component, which is marginalised, and a present but empty sub-data set as observed with zero events. RooFit has no notion of an unobserved component: a combined RooDataSet without entries for a channel is a present and empty channel, and both evaluation backends then add the expected number of events of an extended component to the likelihood (#24004).

When a likelihood pairs a combined_dist with a combined data set, the import now checks the keys:

Case	Behaviour
extended component without sub-data set	error (the fit would differ from what the document means)
non-extended component without sub-data set	warning (changes nothing)
sub-data set without component	warning (RooFit ignores the data)
different columns	error

Test: CombinedDataPairing.

3b. extended on a combined_dist

HS3 #119 states that a combined_dist must not have the component extended: whether it is extended follows from its components. The importer rejects it with an error, whatever its value (test: CombinedDistImportErrors). RooFit never writes it.

4. category_conditional_dist

RooFit has this distribution as the "switch mode" of RooSimultaneous: with an index category that is not an observable, it evaluates to the component selected by the current state, and its expected number of events is the one of that component.

Direction	Change
Import	A category_conditional_dist is a RooSimultaneous over its categorical parameters (declared by categorical axes of a domain), tagged with the attribute hs3_category_conditional_dist. RooFit decides between joint and conditional by the normalization set, i.e. by context; the tag makes the export context-free.
Export	A RooSimultaneous with the tag is written as category_conditional_dist, with categorical_parameter instead of columns, and its analysis is paired with data like any other pdf. A RooSimultaneous built in RooFit is a combined_dist unless the user sets the attribute.
Export	The value of a RooCategory in a snapshot is written to the parameter point as its label (categorical parameters). The columns of combined distributions are not parameters and are left out.

Test: CategoryConditionalDist.

5. Fix: empty weighted data sets

An empty weighted data set over a variable without bins was detected as binned data with zero bins. Unbinned data sets imported from HS3 leave their axes without binning, so a present but empty sub-data set was not stable under round trips. The detection now requires a binning.

6. Components must not share observables

HS3 #119 requires that the components of a combined distribution share no variate, and that the sub-data sets of a combined data set have distinct axis names: a quantity measured in several components is a separate random variable in each of them (e.g. mass_SR and mass_CR), since it follows a different distribution there. Exporting a data set that is split by a RooSimultaneous whose components share an observable is therefore an error, which asks to give the observable a separate name per component. A document whose sub-data sets share an axis is still read, with a warning. Tests: CombinedComponentsMustNotShareObservables; CombinedDataSeveralColumns now uses one observable per component.

7. Data sets with categories but without a combined distribution

A combined data set only makes sense paired with a combined distribution. Without a matching RooSimultaneous, the category of a data set is an ordinary observable, so such a data set is now written as a single data set with a categorical axis (the form of HS3 #108, read since #24001), binned or unbinned. Before, it was split by the category into sub-data sets that all carried the same observables, which #119 forbids. Test: the data-only part of CombinedData.

8. Option: rename the observables of combined data sets

Workspaces whose RooSimultaneous components share an observable (common in older workspaces and in CMS Combine) can't be exported because of 6. The new option RooJSONFactoryWSTool::config().renameCombinedObservables (off by default) renames the observables of every combined data set per key on export: obs_<key> (the labels of the key joined by _), or obs_<key>_<observable> if a component has several observables. The components of the matching RooSimultaneous are written with the renamed observables: they are rebuilt with RooCustomizer, which clones every node depending on a renamed observable with the suffix _<key>, so nodes shared by several components get one copy per component (e.g. shift(mass) → shift_SR(obs_SR), shift_CR(obs_CR)). Observables that already have their new name are left alone, so a second export of an imported document gives the same document. Test: RenameCombinedObservables (the likelihood is the same after the round trip).

Not covered, deferred

• The categorical function of Master tmva dataloader vi #119 and RooMultiPdf/RooMultiReal (both select by category index = list position, RooMultiPdf is never extended and adds a penalty term): #24006.
• Unobserved components (see 3): RooFit would need a way to mark a channel of a combined data set as unobserved (#24004).
• A RooSuperCategory whose inputs are not RooCategorys (nested super categories): explicit error.

Behaviour changes users may notice

• misc.ROOT_internal.combined_distributions and combined_datasets are no longer written; the standard objects replace them. Files with the old form are still read.
• The likelihood of a ModelConfig with a RooSimultaneous lists one pair (the RooSimultaneous and the combined data set) instead of the components. Analyses are named after their ModelConfig (<ModelConfig>_<data>), like all analyses since [RF][HS3] Bring the HS3 import and export closer to the HS3 standard #23996.
• After import, the ModelConfig observables include the index category.
• Category indices become the positions in the domain's categories after a round trip.
• The sub-data sets of a combined data set follow its combined entry in data (they used to be written before all other data sets).
• Values of categorical parameters appear in parameter points as strings.
• Import errors and warnings for the pairing of combined objects (see 3).
• A RooSimultaneous whose components share an observable can no longer be exported together with its data set, unless renameCombinedObservables is enabled (see 6 and 8).
• A data set with a category observable but without a matching RooSimultaneous is written as one data set with a categorical axis, not as sub-data sets (see 7).

Checklist:

• tested changes locally
• updated the docs (if necessary)

Tests run (Linux, Ubuntu 26.04, GCC 15.2, RelWithDebInfo, -Droofit=ON -Dtesting=ON): testRooFitHS3 115 → 126 tests (11 new), testJSONInterface 1, testHistFactory 69 → 70 (1 new), all passing; each commit built and tested on its own. clang-format clean.

AI assistance: Claude Code implemented the changes, the tests and this description under review of the author: claude-fable-5-1 the first 6 commits (sections 1–5), claude-opus-5-5 the last 4 (sections 3b, 6–8). The commits carry the corresponding Assisted-by trailer.

🤖 Generated with Claude Code