[RF] Remove the legacy evaluation backend

.github/workflows/root-ci-config/buildconfig/global.txt

@@ -83,7 +83,7 @@ sqlite=ON
 ssl=ON
 test_distrdf_dask=ON
 test_distrdf_pyspark=ON
-test_roofit_hs3testsuite=ON
+test_roofit_hs3testsuite=OFF
 test_tmva_sofie=ON
 testing=ON
 tmva-cpu=ON

README/ReleaseNotes/v644/index.md

@@ -18,6 +18,7 @@ The following people have contributed to this new version:
 * The `ROOT.SetHeuristicMemoryPolicy` method has been removed. This was introduced in 6.40 as a temporary debugging tool to aid users in addressing the changes in memory ownership heuristics.
 * A `RooArgSet` cannot be constructed with an implicit conversion from Python `set` anymore. The problem is that Python sets are unordered while RooArgSets are ordered, and this mismatch can lead to subtle problems later on. Prefer conversion from Python lists or tuples, which are ordered too.
 * The `RooFit::BatchMode` command argument has been removed, use `RooFit::EvalBackend` instead.
+* The legacy evaluation backend of RooFit was deprecated in ROOT 6.42 and is now removed. See the RooFit section below for details.
 * The `RooTrace` class was untested and is now removed. For memory debugging, please use general-purpose tools such as AddressSanitizer or Valgrind instead.
 * Flattening nested **RooSimultaneous** in the constructor, deprecated in ROOT 6.42, is removed. RooSimultaneous components are now kept as they are, which also allows adding them with `RooSimultaneous::addPdf()`. They are only supported if their index category is a parameter ("switch mode"), which now also works for toy generation and for plotting slices of the outer RooSimultaneous. If the index category of a nested RooSimultaneous is in the dataset, creating the likelihood fails with a clear error for all evaluation backends. To combine channels, use one RooSimultaneous over a **RooCategory** with one state per channel, or over a **RooSuperCategory** if the channels span the full product of several categories. RooSimultaneous objects that were flattened by previous ROOT versions and are read from files are not affected. The internal `RooSimultaneous::InitializationOutput` struct and `RooSimultaneous::initialize()` function are removed as well.
 * The PyMVA library and the associated `tmva-pymva` build option that were deprecated in ROOT 6.42 have been removed
@@ -43,6 +44,40 @@ After ROOT 6.44, the minimum may rise to 3.31 if an upgrade to LLVM 24 or later
 
 ## RooFit
 
+### Removal of the legacy evaluation backend
+
+The `legacy` evaluation backend for likelihood and chi-square fits is removed.
+It was superseded by the vectorized `cpu` backend, which is the default since
+ROOT 6.32. After the removal of the legacy constant term optimization in ROOT 6.42, the
+legacy backend also had no performance-relevant feature left that would justify
+its continued maintenance.
+
+Concretely, this means:
+
+  * `RooFit::EvalBackend::Legacy()` and the corresponding enum value are
+    removed. Passing `RooFit::EvalBackend("legacy")` to `fitTo()`,
+    `createNLL()`, `chi2FitTo()` or `createChi2()` now throws an exception.
+  * The implementation classes of the legacy test statistics are removed:
+    **RooNLLVar**, **RooChi2Var**, **RooAbsOptTestStatistic** and
+    **RooAbsTestStatistic**. Their headers were not part of the public
+    interface anymore since ROOT 6.32, but they were still installed for
+    backwards compatibility.
+  * The old multiprocessing mechanism of the legacy backend is removed as well,
+    consisting of the **RooRealMPFE** class and the underlying
+    **BidirMMapPipe**. The `RooFit::NumCPU()` command argument no longer forks
+    off one `RooRealMPFE` process per CPU: it now selects the number of worker
+    threads for the multi-threaded batch evaluation in the RooBatchCompute
+    library, and its interleaving strategy argument is ignored. For fits with a
+    parallelized gradient, there is also the `RooFit::Parallelize()` argument,
+    based on the `RooFit::MultiProcess` framework.
+  * The `nll::name[pdf,data]` and `chi2::name[pdf,data]` expressions in the
+    `RooWorkspace::factory()` language are removed, since they instantiated the
+    removed classes directly. Use `RooAbsPdf::createNLL()` or
+    `RooAbsReal::createChi2()` instead.
+  * The `RooFit::TestStatistics::RooUnbinnedL` class now always evaluates with
+    the `RooFit::Evaluator` and its `evalBackend` constructor parameter
+    defaults to the `cpu` backend, like `RooFit::TestStatistics::NLLFactory`.
+
 ## Graphics and GUI
 
 ## Geometry

roofit/CMakeLists.txt

@@ -4,8 +4,6 @@
 # For the licensing terms see $ROOTSYS/LICENSE.
 # For the list of contributors see $ROOTSYS/README/CREDITS.
 
-set(roofit_legacy_eval_backend ON CACHE BOOL "" FORCE)
-
 add_subdirectory(batchcompute)
 add_subdirectory(codegen)
 if(NOT WIN32)
@@ -19,7 +17,7 @@ endif()
 add_subdirectory(roostats)
 add_subdirectory(histfactory)
 add_subdirectory(hs3)
-if(roofit_legacy_eval_backend AND NOT MSVC)
+if(NOT MSVC)
   add_subdirectory(xroofit)
 endif()
 

roofit/histfactory/test/testHistFactory.cxx

@@ -40,7 +40,7 @@ namespace {
 // If the JSON files should be written out for debugging purpose.
 const bool writeJsonFiles = false;
 
-std::vector<double> getValues(RooAbsReal const &real, RooRealVar &obs, bool normalize, bool useBatchMode)
+std::vector<double> getValues(RooAbsReal const &real, RooRealVar &obs, bool normalize)
 {
    RooArgSet normSet{obs};
 
@@ -53,10 +53,6 @@ std::vector<double> getValues(RooAbsReal const &real, RooRealVar &obs, bool norm
       out.push_back(normalize ? real.getVal(normSet) : real.getVal());
    }
 
-   if (useBatchMode == false) {
-      return out;
-   }
-
    std::unique_ptr<RooAbsReal> clone;
    if (normalize) {
       clone = RooFit::Detail::compileForNormSet<RooAbsReal>(real, obs);
@@ -444,7 +440,6 @@ TEST_P(HFFixtureEval, Evaluation)
    const double systEps = 1e-6;
 
    const MakeModelMode makeModelMode = std::get<0>(GetParam());
-   const bool useBatchMode = std::get<2>(GetParam()) != RooFit::EvalBackend(RooFit::EvalBackend::Value::Legacy);
 
    RooHelpers::HijackMessageStream evalMessages(RooFit::INFO, RooFit::FastEvaluations);
 
@@ -460,8 +455,8 @@ TEST_P(HFFixtureEval, Evaluation)
    ASSERT_NE(mc, nullptr);
 
    // Test evaluating the model:
-   std::vector<double> normResults = getValues(*channelPdf, *obs, true, useBatchMode);
-   std::vector<double> unnormResults = getValues(*channelPdf, *obs, false, useBatchMode);
+   std::vector<double> normResults = getValues(*channelPdf, *obs, true);
+   std::vector<double> unnormResults = getValues(*channelPdf, *obs, false);
 
    for (int i = 0; i < obs->numBins(); ++i) {
       EXPECT_NEAR(unnormResults[i], _tgtNom[i] / obs->getBinWidth(i), defaultEps);
@@ -477,17 +472,17 @@ TEST_P(HFFixtureEval, Evaluation)
 
       // Test syst up:
       var->setVal(1.);
-      std::vector<double> normResultsSyst = getValues(*channelPdf, *obs, true, useBatchMode);
-      std::vector<double> unnormResultsSyst = getValues(*channelPdf, *obs, false, useBatchMode);
+      std::vector<double> normResultsSyst = getValues(*channelPdf, *obs, true);
+      std::vector<double> unnormResultsSyst = getValues(*channelPdf, *obs, false);
       for (int i = 0; i < obs->numBins(); ++i) {
          EXPECT_NEAR(unnormResultsSyst[i], _tgtSysUp[i] / obs->getBinWidth(i), systEps);
          EXPECT_NEAR(normResultsSyst[i], _tgtSysUp[i] / obs->getBinWidth(i) / (_tgtSysUp[0] + _tgtSysUp[1]), systEps);
       }
 
       // Test syst down:
       var->setVal(-1.);
-      normResultsSyst = getValues(*channelPdf, *obs, true, useBatchMode);
-      unnormResultsSyst = getValues(*channelPdf, *obs, false, useBatchMode);
+      normResultsSyst = getValues(*channelPdf, *obs, true);
+      unnormResultsSyst = getValues(*channelPdf, *obs, false);
       for (int i = 0; i < obs->numBins(); ++i) {
          EXPECT_NEAR(unnormResultsSyst[i], _tgtSysDo[i] / obs->getBinWidth(i), systEps);
          EXPECT_NEAR(normResultsSyst[i], _tgtSysDo[i] / obs->getBinWidth(i) / (_tgtSysDo[0] + _tgtSysDo[1]), systEps);

roofit/hs3/test/hs3testsuite_roofit_backend.py

@@ -52,11 +52,7 @@ def run_twice_delta_nll_scan(self, workspace, check: dict[str, Any]) -> list[flo
 
         self._apply_parameter_point(workspace, check["reference_point"])
         with suppress_root_output():
-            nll = pdf.createNLL(
-                data,
-                self.ROOT.RooFit.NumCPU(1),
-                self.ROOT.RooFit.EvalBackend("legacy"),
-            )
+            nll = pdf.createNLL(data)
             reference = float(nll.getVal())
         values = []
         scan_parameter = check["scan_parameter"]

roofit/roofit/test/testRooIntegralMorph.cxx

@@ -251,8 +251,8 @@ TEST(RooIntegralMorph, AlphaCacheScan)
 
 /// Generating a toy dataset from the morph pdf and fitting it back must
 /// recover the true alpha, with the alpha cache enabled as in a realistic
-/// fitting application, on both the legacy and cpu evaluation backends. This
-/// covers the toy fit of the former stressRooFit test 705.
+/// fitting application. This covers the toy fit of the former stressRooFit
+/// test 705.
 TEST(RooIntegralMorph, GenerateAndFit)
 {
    GaussPolySetup s;
@@ -270,7 +270,7 @@ TEST(RooIntegralMorph, GenerateAndFit)
 
    morph.setCacheAlpha(true);
 
-   for (std::string backend : {"legacy", "cpu"}) {
+   for (std::string backend : {"cpu"}) {
       alpha.setVal(0.5);
       alpha.setError(0.0);
       std::unique_ptr<RooFitResult> res{

roofit/roofit/test/vectorisedPDFs/VectorisedPDFTests.cxx

@@ -420,35 +420,6 @@ void PDFTest::checkParameters()
    }
 }
 
-void PDFTest::runBatchVsScalar(bool clonePDF)
-{
-   RooAbsPdf *pdfScalar = _pdf.get();
-   RooAbsPdf *pdfBatch = _pdf.get();
-   std::unique_ptr<RooAbsPdf> cleanupScalar;
-   std::unique_ptr<RooAbsPdf> cleanupBatch;
-
-   if (clonePDF) {
-      pdfScalar = static_cast<RooAbsPdf *>(_pdf->cloneTree("PDFForScalar"));
-      pdfBatch = static_cast<RooAbsPdf *>(_pdf->cloneTree("PDFForScalar"));
-
-      cleanupScalar.reset(pdfScalar);
-      cleanupBatch.reset(pdfBatch);
-   }
-
-   resetParameters();
-   auto resultScalar = runScalarFit(pdfScalar);
-
-   resetParameters();
-   auto resultBatch = runBatchFit(pdfBatch);
-
-   resetParameters();
-
-   ASSERT_NE(resultScalar, nullptr);
-   ASSERT_NE(resultBatch, nullptr);
-
-   EXPECT_TRUE(resultScalar->isIdentical(*resultBatch, _toleranceParameter, _toleranceCorrelation));
-}
-
 std::unique_ptr<RooFitResult> PDFTest::runBatchFit(RooAbsPdf *pdf)
 {
    if (!_dataFit)
@@ -493,50 +464,6 @@ std::unique_ptr<RooFitResult> PDFTest::runBatchFit(RooAbsPdf *pdf)
    return result;
 }
 
-std::unique_ptr<RooFitResult> PDFTest::runScalarFit(RooAbsPdf *pdf)
-{
-   if (!_dataFit)
-      makeFitData();
-
-   kickParameters();
-   makePlots(::testing::UnitTest::GetInstance()->current_test_info()->name() + std::string("_scalar_prefit"));
-
-   std::unique_ptr<RooArgSet> pars{pdf->getParameters(*_dataFit)};
-   pars->assign(_parameters);
-
-   for (unsigned int index = 0; index < pars->size(); ++index) {
-      auto pdfParameter = static_cast<RooAbsReal *>((*pars)[index]);
-      auto origParameter = static_cast<RooAbsReal *>(_origParameters.find(*pdfParameter));
-      if (!origParameter || origParameter->isConstant())
-         continue;
-
-      EXPECT_NE(pdfParameter->getVal(), origParameter->getVal())
-         << "Parameter #" << index << "=" << pdfParameter->GetName() << " is identical after kicking.";
-   }
-
-   if (HasFailure()) {
-      std::cout << "Pre-fit parameters:\n";
-      _parameters.Print("V");
-      std::cout << "Orig parameters:\n";
-      _origParameters.Print("V");
-   }
-
-   MyTimer singleTimer("Fitting scalar mode " + _name);
-   std::unique_ptr<RooFitResult> result{pdf->fitTo(*_dataFit, RooFit::EvalBackend(RooFit::EvalBackend::Value::Legacy), RooFit::SumW2Error(false),
-                                                   RooFit::PrintLevel(_printLevel), RooFit::Save(),
-                                                   _multiProcess > 0 ? RooFit::NumCPU(_multiProcess) : RooCmdArg())};
-   std::cout << singleTimer;
-   EXPECT_NE(result, nullptr);
-   if (!result)
-      return nullptr;
-
-   EXPECT_EQ(result->status(), 0) << "[Scalar fit did not converge.]";
-
-   makePlots(::testing::UnitTest::GetInstance()->current_test_info()->name() + std::string("_scalar_postfit"));
-
-   return result;
-}
-
 void PDFTestWeightedData::makeFitData()
 {
    PDFTest::makeFitData();

roofit/roofit/test/vectorisedPDFs/VectorisedPDFTests.h

@@ -51,12 +51,9 @@ class PDFTest : public ::testing::Test {
 
    void checkParameters();
 
-   void runBatchVsScalar(bool clonePDF = false);
 
    std::unique_ptr<RooFitResult> runBatchFit(RooAbsPdf *pdf);
 
-   std::unique_ptr<RooFitResult> runScalarFit(RooAbsPdf *pdf);
-
    std::unique_ptr<RooAbsPdf> _pdf;
    std::unique_ptr<RooDataSet> _dataUniform;
    std::unique_ptr<RooDataSet> _dataFit;
@@ -166,32 +163,3 @@ class PDFTestWeightedData : public PDFTest {
       checkParameters();                      \
    }
 
-#ifdef ROOFIT_LEGACY_EVAL_BACKEND
-
-/// Run a fit for batch and scalar code and compare results.
-#define FIT_TEST_BATCH_VS_SCALAR(TEST_CLASS, TEST_NAME) \
-   TEST_F(TEST_CLASS, TEST_NAME) { runBatchVsScalar(); }
-
-/// Run a fit for batch and scalar code and compare results.
-/// Clone the PDFs before running the tests. This can run the test even if some internal state
-/// is propagated / saved wrongly.
-#define FIT_TEST_BATCH_VS_SCALAR_CLONE_PDF(TEST_CLASS, TEST_NAME) \
-   TEST_F(TEST_CLASS, TEST_NAME) { runBatchVsScalar(true); }
-
-/// Run a fit in legacy mode and compare results to pre-fit values.
-#define FIT_TEST_SCALAR(TEST_CLASS, TEST_NAME) \
-   TEST_F(TEST_CLASS, TEST_NAME)               \
-   {                                           \
-      auto result = runScalarFit(_pdf.get());  \
-      ASSERT_NE(result, nullptr);              \
-      checkParameters();                       \
-   }
-
-#else
-
-// Ignore legacy tests if legacy backend is not available
-#define FIT_TEST_BATCH_VS_SCALAR(TEST_CLASS, TEST_NAME)
-#define FIT_TEST_BATCH_VS_SCALAR_CLONE_PDF(TEST_CLASS, TEST_NAME)
-#define FIT_TEST_SCALAR(TEST_CLASS, TEST_NAME)
-
-#endif

roofit/roofit/test/vectorisedPDFs/testAddPdf.cxx

@@ -67,10 +67,7 @@ COMPARE_FIXED_VALUES_UNNORM(TestGaussPlusPoisson, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestGaussPlusPoisson, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestGaussPlusPoisson, CompareFixedValuesNormLog)
 
-FIT_TEST_SCALAR(TestGaussPlusPoisson, DISABLED_Scalar) // Save time
 FIT_TEST_BATCH(TestGaussPlusPoisson, DISABLED_Batch)   // Save time
-FIT_TEST_BATCH_VS_SCALAR(TestGaussPlusPoisson, CompareBatchScalar)
-
 class TestGaussPlusGaussPlusExp : public PDFTest {
 protected:
    TestGaussPlusGaussPlusExp() : PDFTest("Gauss + Gauss + Exp")
@@ -126,10 +123,7 @@ COMPARE_FIXED_VALUES_UNNORM(TestGaussPlusGaussPlusExp, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestGaussPlusGaussPlusExp, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestGaussPlusGaussPlusExp, CompareFixedValuesNormLog)
 
-FIT_TEST_SCALAR(TestGaussPlusGaussPlusExp, DISABLED_Scalar) // Save time
 FIT_TEST_BATCH(TestGaussPlusGaussPlusExp, DISABLED_Batch)   // Save time
-FIT_TEST_BATCH_VS_SCALAR(TestGaussPlusGaussPlusExp, CompareBatchScalar)
-
 #if !defined(_MSC_VER) // RooFit multiprocessing doesn't work on Windows
 
 class TestGaussPlusGaussPlusExp_MP : public TestGaussPlusGaussPlusExp {
@@ -141,8 +135,5 @@ COMPARE_FIXED_VALUES_UNNORM(TestGaussPlusGaussPlusExp_MP, CompareFixedValuesUnno
 COMPARE_FIXED_VALUES_NORM(TestGaussPlusGaussPlusExp_MP, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestGaussPlusGaussPlusExp_MP, CompareFixedValuesNormLog)
 
-FIT_TEST_SCALAR(TestGaussPlusGaussPlusExp_MP, DISABLED_Scalar) // Save time
 FIT_TEST_BATCH(TestGaussPlusGaussPlusExp_MP, DISABLED_Batch)   // Save time
-FIT_TEST_BATCH_VS_SCALAR(TestGaussPlusGaussPlusExp_MP, CompareBatchScalar)
-
 #endif // !defined(_MSC_VER)

roofit/roofit/test/vectorisedPDFs/testArgusBG.cxx

@@ -46,6 +46,4 @@ class TestArgus : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestArgus, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestArgus, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestArgus, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestArgus, RunScalar)
 FIT_TEST_BATCH(TestArgus, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestArgus, CompareBatchScalar)

roofit/roofit/test/vectorisedPDFs/testBernstein.cxx

@@ -41,10 +41,7 @@ class TestBernstein2 : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestBernstein2, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestBernstein2, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestBernstein2, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestBernstein2, RunScalar)
 FIT_TEST_BATCH(TestBernstein2, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestBernstein2, CompareBatchScalar)
-
 class TestBernstein5 : public PDFTest {
 protected:
    TestBernstein5() : PDFTest("Bernstein5")
@@ -74,6 +71,4 @@ class TestBernstein5 : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestBernstein5, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestBernstein5, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestBernstein5, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestBernstein5, RunScalar)
 FIT_TEST_BATCH(TestBernstein5, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestBernstein5, CompareBatchScalar)

roofit/roofit/test/vectorisedPDFs/testBifurGauss.cxx

@@ -45,6 +45,4 @@ class TestBifurGauss : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestBifurGauss, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestBifurGauss, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestBifurGauss, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestBifurGauss, RunScalar)
 FIT_TEST_BATCH(TestBifurGauss, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestBifurGauss, CompareBatchScalar)

roofit/roofit/test/vectorisedPDFs/testBreitWigner.cxx

@@ -42,6 +42,4 @@ class TestBreitWigner : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestBreitWigner, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestBreitWigner, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestBreitWigner, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestBreitWigner, RunScalar)
 FIT_TEST_BATCH(TestBreitWigner, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestBreitWigner, CompareBatchScalar)

roofit/roofit/test/vectorisedPDFs/testBukin.cxx

@@ -56,6 +56,4 @@ class TestBukin : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestBukin, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestBukin, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestBukin, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestBukin, RunScalar)
 FIT_TEST_BATCH(TestBukin, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestBukin, CompareBatchScalar)

roofit/roofit/test/vectorisedPDFs/testCBShape.cxx

@@ -48,6 +48,4 @@ class TestCBShape : public PDFTest {
 COMPARE_FIXED_VALUES_UNNORM(TestCBShape, CompareFixedValuesUnnorm)
 COMPARE_FIXED_VALUES_NORM(TestCBShape, CompareFixedValuesNorm)
 COMPARE_FIXED_VALUES_NORM_LOG(TestCBShape, CompareFixedNormLog)
-FIT_TEST_SCALAR(TestCBShape, RunScalar)
 FIT_TEST_BATCH(TestCBShape, RunBatch)
-FIT_TEST_BATCH_VS_SCALAR(TestCBShape, CompareBatchScalar)