Stage 4: never replace a test file's #include list with an empty one

[mkarlesky]

Summary

Stage 4 could replace a test file's #include list with an empty one. The test then lost its mocks and sources, and the build stopped on a compile error about a mock (implicit declaration of function 'Gpt_StartTimer_Expect'). Nothing at default verbosity explained it. Analysis: https://claude.ai/artifact/TR6uCuBLbfdWz9oUNRPcVv

It happened two ways:

• Both passes failed for one file. Pass 1 reported a failure (no make rule) as [] and cached it as fresh, so one killed or misconfigured run stuck until the test file changed. With pass 2 also failed, reconciling against that [] wiped the list. Since Fallback preprocessing: reconcile test #includes against a plain expansion #1346 this applies in fallback too.
• -P / --no-line-commands reached pass 2. These strip the line markers pass 2 is read through. They also break mock header preprocessing, and stage 11 then finds zero test cases.

Changes

• A failed pass 1 is nil, not [].

  • It is logged at obnoxious verbosity with the tool's output.
  • Stage 4 doesn't cache it, so the next run retries, and an empty list stands in meanwhile.
  • A mockable header's or Partial's text scan stands in the same way. Its own cache still keeps the result, which is documented as a best-effort limit.

• Both passes found nothing. Stage 2's text-scan list stands as it is, as a best effort, with the Partial headers its configuration added.

• Guard. Reconciliation never replaces a non-empty text-scan list with an empty one. Stage 2's list stays, with a COMPLAIN warning naming the file.

• Validation. With test preprocessing on, startup validation now rejects -P and --no-line-commands:

  • in the :preprocess and :compile flags of :test and plugin contexts, both array and matcher shapes (:compile flags reach preprocessing by default);
  • in the full and directives-only preprocessor tools' :arguments, naming a :tools_<name> shortcut when that is where the entry was written.

  The new stateless LineMarkerSuppressors finds the entries, and ConfiguratorSetup#validate_line_markers reports each by its configuration path.

• Refactor. extract_bare_includes and the text scan are broken up. The handler now meets AbcSize and MethodLength, and the RuboCop backlog drops from 838 to 835.

• Docs. PIPELINE.md and preprocess/README.md are updated. The README's "deduplicated by filename" is corrected to filepath.

Tests (7-step test-first, no new system specs)

• Pins:
  • a rule naming no #includes gives a cached [];
  • an empty reconciled list is ingested when the text scan is empty too;
  • a failing pass 1 alone still reconciles normally.
• Red, then green:
  • Handler: nil and its log.
  • Header path: survives nil.
  • Stage 4: no cache on failure.
  • Both failed: keeps the text scan, in both modes.
  • Guard: header and source-only cases.
  • LineMarkerSuppressors: shapes, -Wp,, joined entries, no false positives, malformed input, shortcut attribution.
  • validate_line_markers: including :none skipping the check.
• Integration (real GCC). The harnesses gained bare_tool: and flags:. FAILING_BARE_TOOL (an unrecognized option) reproduces a pass 1 failure. Examples run in both modes:
  • both passes failing keeps the mocks;
  • a failed pass 1 writes no cache;
  • -P hits the guard;
  • a header with a failing bare tool equals the success result;
  • configurator_pipeline_spec rejects -P and a shortcut's --no-line-commands, and accepts -P under :none.
• Runs.
  • Units (4716) and integration (480) pass on Apple Clang 15. Integration passes on GCC 14 in Docker.
  • System regressions pass in Docker: conditional_mock_includes, conditional_include_macro_visibility, preprocessing_fallback_conditionals, delta_builds, partials_module_disambiguation, same_named_modules, include_path_reconciliation, test_source_file_directive.
• RuboCop: 835, with no new file in the backlog. lint:changed is clean.

Notes for review

• Guard false positive. A test file whose every #include sits in a branch that is off now keeps its text scan and warns, where it used to get the correct empty list. That needs no unconditional #include at all, not even unity.h.
• Both-failed path. Keeping the text scan means mocks in branches that are off stay. That is the best-effort trade.
• Breaking change. Rejecting -P breaks a project that sets it with test preprocessing on. Such builds already fail or run zero tests.
• Release docs. Changelog, KnownIssues, BreakingChanges and mkdocs edits will follow as uncommitted diffs after CI.

🤖 Generated with Claude Code