feat: one status per screenshot; record: :all compares before re-recording

CHANGELOG.md

@@ -179,8 +179,8 @@ and assigning `nil` hands the setting back to the environment.
 - **Accepting an intentional change is one setting, not folklore.** `record` replaces
   `fail_if_new` (removed in 2.1): `:once` records a missing baseline and compares the rest —
   what a local run already did; `:none` **fails** on a missing baseline with the `git add`
-  command attached, and is what CI gets by default; `:all` re-records every screenshot and
-  compares nothing, the bulk-accept verb for the run after an intentional redesign. 1.15.1
+  command attached, and is what CI gets by default; `:all` compares like `:once` but accepts
+  what differs instead of failing, the bulk-accept verb for the run after an intentional redesign. 1.15.1
   documented `RECORD_SCREENSHOTS=1` in four files and printed it in its own error message,
   and nothing in the gem ever read that variable — accepting a change was answered in issue
   comments for six years and never in the docs. See `docs/configuration.md` (#259)
@@ -331,17 +331,35 @@ One removal 2.0 cannot warn about, written down instead: the legacy `LOADED_DRIV
   the name to install; the mirror reserves the forward-looking one. **Install one,
   never both.**
 - **Reporters can list the screenshots that were not compared.**
-  `SnapDiff::Reporting.uncompared` returns the ones behind the summary line's "new" and
-  "re-recorded" counts, complete by the time reporters `finalize` (fork-parallel workers
-  included): each with its `name`, the captured file's `path`, and the `reason` — `:new`
-  (no committed baseline) or `:rerecorded` (`record: :all`). The bundled HTML report lists
-  new screenshots from it
+  `SnapDiff::Reporting.uncompared` returns the ones behind the summary line's "new"
+  count, complete by the time reporters `finalize` (fork-parallel workers included): each
+  with its `name`, the captured file's `path`, and the `status` (`:new`, no committed
+  baseline). The bundled HTML report lists new screenshots from it
 - New documentation shipped inside the gem:
   [docs/snapdiff.md](docs/snapdiff.md) (canonical API) and
   [docs/UPGRADING.md](docs/UPGRADING.md) (every renamed constant, which names warn,
   rollback)
 
 ### Changed
+- **`record: :all` now compares before re-recording and the report shows what changed.**
+  A screenshot that differs from its committed baseline is accepted as the new baseline
+  (it never fails the run) and leaves the usual `.base.png` and diff/heatmap files,
+  which the recommended `.gitignore` already ignores; a screenshot that matches is left
+  as committed. The summary line counts the accepted ones in a parenthesis —
+  `12 verified, 3 changed (3 re-recorded), 0 new (not verified).` — and the HTML report
+  lists each of them with its baseline, current and heatmap views and a "re-recorded"
+  badge, and its title and pill use the same `N changed (R re-recorded)` wording as the
+  terminal (the title used to say "failures"). Report entries gain a `status` (`changed`,
+  `new` or `rerecorded`) in place of `new_screenshot`; an entry the report could not
+  render also carries an `error`.
+- **Reporters now receive every screenshot, each with a `status`.** New and re-recorded
+  (`record: :all`) captures used to bypass them; now every capture is a
+  `SnapDiff::ScreenshotAssertion` whose `status` is `:passed`, `:changed`, `:new` or
+  `:rerecorded` (`compare` is nil when there was no baseline). Reading `status` never
+  forces a comparison. A custom reporter that treated "has an assertion" as "was
+  compared" should test `assertion.status == :changed`, as the documented idiom and
+  `AISimple` now do. `pending_if_new` now also sees a new screenshot under `record: :none`,
+  where it is registered before the "no existing screenshot" failure, not only under `:once`
 - **No `activesupport` at runtime.** 1.15.1's `snap_manager.rb` required
   `active_support/core_ext/module/attribute_accessors` while `activesupport` was only a
   *development* dependency, so a bundle without Rails died at
@@ -450,8 +468,7 @@ One removal 2.0 cannot warn about, written down instead: the legacy `LOADED_DRIV
   baseline appears after the failures, marked `new`, with its capture and the
   `git add` that makes it the baseline; a run whose only finding is a new screenshot
   now writes a report. The report's `new` count is the summary line's, including
-  across `parallelize(workers: N)`. Re-recorded screenshots (`record: :all`) are
-  counted in the report but not yet listed
+  across `parallelize(workers: N)`
 - **Under `record: :none` a new screenshot is counted.** The run still fails as before;
   the summary line now says `1 new` instead of `0 new`
 - **A committed baseline taken by a same-name test is reported as disappeared, in every

README.md

@@ -211,15 +211,16 @@ SnapDiff.config.record = :all unless ENV["ACCEPT_SCREENSHOTS"].to_s.empty?
 
 ```bash
 ACCEPT_SCREENSHOTS=1 bin/rails test:system
-# [snap_diff] record: :all re-recorded 40 screenshots WITHOUT comparing: ...
+# [snap_diff] 212 verified, 40 changed (40 re-recorded), 0 new (not verified).
 
 git status                     # forty modified baselines, and nothing else
 git add doc/screenshots/
 git commit -m "chore: re-record baselines after the checkout redesign"
 ```
 
-`:all` writes every capture straight to its baseline path and compares nothing, so `git status`
-is the review surface — look at the images before you commit. It **refuses to run under CI**,
+`:all` compares every capture to its committed baseline like `:once` does, but a screenshot that
+differs is accepted as the new baseline instead of failing the test — the report lists what changed,
+and `git status` is the review surface — look at the images before you commit. It **refuses to run under CI**,
 where there would be nobody to look. The other modes are `:once` (the default) and `:none`
 (strict — a missing baseline always fails); see
 [Record modes](docs/configuration.md#record-modes--accepting-changes).

docs/UPGRADING.md

@@ -319,7 +319,7 @@ actually want, which is *accept this change*. 2.0 adds the verb:
 ```ruby
 SnapDiff.config.record = :once   # default. Record a screenshot that has no baseline.
 SnapDiff.config.record = :none   # strict. A missing baseline always fails.
-SnapDiff.config.record = :all    # re-record everything. THE BULK-ACCEPT MODE.
+SnapDiff.config.record = :all    # compare, and accept what differs. THE BULK-ACCEPT MODE.
 ```
 
 | Old | New | Note |
@@ -342,8 +342,8 @@ missing-baseline default is deliberately unchanged; `:none` is how you opt into
 explicitly. **Precedence:** an explicitly set mode outranks `fail_if_new`, and `fail_if_new`
 decides only when no mode was set — the same rule `fail_if_new` itself has over the `CI` sniff.
 
-`record = :all` **refuses to run under CI**: it accepts every rendering by design, so left in a
-committed config file it would be a build that compares nothing and passes forever. See
+`record = :all` **refuses to run under CI**: it accepts every difference by design, so left in a
+committed config file it would be a build that never fails. See
 [Record modes](configuration.md#record-modes--accepting-changes).
 
 #### Silent by design

docs/architecture.md

@@ -186,14 +186,15 @@ Test begins
     │
     ├─ test body:
     │    └─ screenshot("name")
-    │         ├─ ScreenshotMatcher builds assertion
-    │         ├─ add to Thread-local registry (always)
-    │         └─ delayed=false → also validate immediately
+    │         ├─ ScreenshotMatcher builds an assertion for every capture
+    │         │    (compared, new, or re-recorded) and registers it in the
+    │         │    Thread-local registry — before any raise
+    │         └─ delayed=false → the DSL also validates it immediately
     │
     ├─ teardown:
     │    ├─ SnapDiff.session.verify
     │    │    ├─ iterates the fiber-local assertions
-    │    │    ├─ calls validate on each
+    │    │    ├─ calls validate on each (new and re-recorded ones do not fail)
     │    │    └─ raises SnapDiff::ExpectationNotMet if any differed
     │    │
     │    └─ SnapDiff.reset (always, in an ensure)

docs/ci-integration.md

@@ -273,7 +273,7 @@ jobs:
 > than relying on the files being there.
 >
 > **`record = :all` is not the mode for this job** — it refuses to run under CI, because it
-> would accept every *changed* rendering unreviewed as well. `:once` records what is new and
+> would accept every *changed* rendering unreviewed and never fail. `:once` records what is new and
 > keeps comparing everything that already has a baseline.
 
 **How it works:**

docs/configuration.md

@@ -63,7 +63,7 @@ SnapDiff.config.record = :once   # what a local run already does; see Precedence
 |------|------------------|------------------|-------------------|
 | `:once` | recorded, not compared | compared | the default — you want a normal run |
 | `:none` | **fails** with the `git add` command attached | compared | every screenshot must already be recorded |
-| `:all` | recorded | **re-recorded, not compared** | you changed the UI on purpose and want the new rendering to become the baseline |
+| `:all` | recorded | compared; **re-recorded on difference, never fails** | you changed the UI on purpose and want the new rendering to become the baseline |
 
 ### `:all` — the bulk-accept verb
 
@@ -82,19 +82,22 @@ SnapDiff.config.record = :all unless ENV["ACCEPT_SCREENSHOTS"].to_s.empty?
 ACCEPT_SCREENSHOTS=1 bin/rails test:system     # with the line above in test_helper.rb
 ```
 
-Every screenshot is written to its baseline path and nothing is compared, so `git status` lists
-exactly what changed. Review the images, then commit them — see
+Every screenshot is compared to its committed baseline, as under `:once`. One that differs is kept
+as the new baseline instead of failing the test, and leaves the usual `.base.png` and diff files
+(ignored by the recommended `.gitignore`); one that matches changes nothing. So `git status` lists
+exactly what changed, and the HTML report shows each change as "re-recorded". Review the images, then commit them — see
 [Accepting an intentional change](../README.md#accepting-an-intentional-change).
 
 At the end of the run the gem names what it accepted:
 
 ```
-[snap_diff] record: :all re-recorded 3 screenshots WITHOUT comparing: checkout/cart, checkout/payment, checkout/review. Review the result before committing -- an unintended change is accepted just as silently.
+[snap_diff] 212 verified, 3 changed (3 re-recorded), 0 new (not verified).
+[snap_diff] record: :all compared 3 screenshots, found them different and accepted them as the new baseline: checkout/cart, checkout/payment, checkout/review. `git add` the file(s) to keep the change, `git restore -- <path>` to discard it. The `.base.<fmt>` and diff files left beside each one are covered by the recommended .gitignore.
 ```
 
 > **`:all` refuses to run under CI** (when `ENV['CI']` is set to a non-empty value). It accepts
-> every rendering by design, so a mode left in a committed config file would buy you a build that
-> compares nothing and passes forever, with the "recorded" screenshots discarded when the runner
+> every difference by design, so a mode left in a committed config file would buy you a build that
+> never fails, with the "recorded" screenshots discarded when the runner
 > is torn down. Re-record locally, where you can look at the result.
 >
 > A CI job that needs to record screenshots with **no baseline yet** does not need `:all`:

docs/images/snap_diff_assertion_flow.html

@@ -139,8 +139,8 @@ <h1>What one <em>assert_matches_screenshot</em> actually does</h1>
       <text x="440" y="644" fill="#9aa0b4" font-size="9" font-family="'Geist Mono', monospace" text-anchor="middle">HTML + summary line</text>
 
       <!-- EDITORIAL ASIDE (1 of max 2) -->
-      <text x="396" y="502" fill="#9aa0b4" font-size="14" font-style="italic" font-family="'Instrument Serif', serif">record: :all sends every screenshot</text>
-      <text x="396" y="522" fill="#9aa0b4" font-size="14" font-style="italic" font-family="'Instrument Serif', serif">down the right-hand path — baseline or not.</text>
+      <text x="396" y="502" fill="#9aa0b4" font-size="14" font-style="italic" font-family="'Instrument Serif', serif">record: :all still compares; it accepts</text>
+      <text x="396" y="522" fill="#9aa0b4" font-size="14" font-style="italic" font-family="'Instrument Serif', serif">a difference instead of failing on it.</text>
 
       <!-- LEGEND -->
       <line x1="30" y1="692" x2="870" y2="692" stroke="rgba(232,234,237,0.10)" stroke-width="0.8"/>

docs/reporters.md

@@ -102,7 +102,7 @@ Build your own reporter by implementing `record`, `finalize` and `summary`:
 class MyReporter
   def record(assertions)
     assertions.each do |assertion|
-      next unless assertion.compare&.difference&.different?
+      next unless assertion.status == :changed
       # process the failure — send to Slack, write JSON, etc.
     end
   end
@@ -130,11 +130,15 @@ the framework's end-of-suite hook (`Minitest.after_run`, RSpec `after(:suite)`,
 finalized and then warned about. A reporter that raises is warned about and skipped — the others
 still run.
 
-`record` receives every assertion, but only a compared one carries a `compare`; the rest have
-`compare: nil` and no path or reason of their own. Those — each new (`reason: :new`) or
-re-recorded (`reason: :rerecorded`) screenshot, with its `name` and captured `path` — are in
-`SnapDiff::Reporting.uncompared`, complete by the time `finalize` runs (fork-parallel workers
-included). The HTML report lists new screenshots from it.
+`record` receives every assertion — one per capture. `assertion.status` says what you hold:
+`:passed` (matches its committed baseline), `:changed` (differs), `:new` (no committed
+baseline) or `:rerecorded` (accepted by `record: :all`). `compare` is nil for a screenshot
+without a baseline; `assertion.path` is the captured file either way. Test `status`, not
+`compare`, to find failures: the tally has already run every comparison by the time `record`
+is called, and reading `status` never forces one. `:rerecorded` was compared and differed,
+like `:changed`, but `record: :all` accepted it. The new screenshots are also in
+`SnapDiff::Reporting.uncompared` (each with `name`, `path` and `status`), complete by the time
+`finalize` runs (fork-parallel workers included); `status` is the contract for everything else.
 
 Full details in [Custom reporters](snapdiff.md#custom-reporters).
 
@@ -146,4 +150,7 @@ default, pluggable backends) and its verdicts show up in this HTML report as
 badges. Advisory by default — it never changes pass/fail unless you opt in
 with `fail_on: %w[real_bug]`. Setup and recipes: [AI triage](ai.md).
 
+It triages `:changed` comparisons only. A `:rerecorded` one was accepted by `record: :all`, so it is
+not analysed, on purpose: a bulk accept should not cost a backend call per screenshot.
+
 [← Back to README](../README.md)

docs/snapdiff.md

@@ -231,7 +231,7 @@ class SlackReporter
   # Called once per finished test, with that test's assertions.
   def record(assertions)
     assertions.each do |assertion|
-      next unless assertion.compare&.difference&.different?
+      next unless assertion.status == :changed
       @failures << assertion.name
     end
   end
@@ -258,16 +258,19 @@ and then warned about (`[snap_diff] Reporter … failed (NoMethodError: undefine
 
 A reporter that raises is warned about and skipped — the other reporters still run.
 
-`record` receives every assertion, but a screenshot that was not compared arrives with
-`compare: nil` and nothing else. Those — the summary line's "new" and "re-recorded" — are in
-`SnapDiff::Reporting.uncompared`, complete by the time `finalize` runs (fork-parallel workers
-included):
+`record` receives every assertion — one per capture. `assertion.status` says what you hold:
+`:passed`, `:changed`, `:new` (no committed baseline) or `:rerecorded` (differs, accepted by `record: :all`).
+`compare` is nil for a screenshot without a baseline; `assertion.path` is the captured file
+either way. The tally has already run every comparison by the time `record` is called, so
+test `status == :changed` rather than digging into `compare`; reading `status` never forces one.
+The new screenshots — the summary line's "new" — are also in `SnapDiff::Reporting.uncompared`,
+complete by the time `finalize` runs (fork-parallel workers included):
 
 ```ruby
   def finalize
-    new_ones = SnapDiff::Reporting.uncompared.select { |s| s.reason == :new }
+    new_ones = SnapDiff::Reporting.uncompared
     # each has #name, #path (the captured file as a Pathname, or nil when it isn't known)
-    # and #reason (:new or :rerecorded); check s.path&.exist? before reading it
+    # and #status (:new); check s.path&.exist? before reading it
   end
 ```
 

lib/snap_diff/config.rb

@@ -190,15 +190,16 @@ def fail_on_difference=(value)
     end
 
     # THE ACCEPT WORKFLOW (#259). What to do about a screenshot whose
-    # baseline is missing -- or, for +:all+, about every screenshot there
-    # is. VCR-shaped: modes in config, because there is no runner to hang a
+    # baseline is missing -- or, for +:all+, about every screenshot that
+    # differs. VCR-shaped: modes in config, because there is no runner to hang a
     # CLI flag on.
     #
     # +:once+:: (default) record a screenshot that has no committed
     #           baseline; compare against the baseline when there is one.
     # +:none+:: strict. A missing baseline always fails.
-    # +:all+::  re-record. Every screenshot is written as the new baseline
-    #           and nothing is compared -- the bulk-accept verb, for the
+    # +:all+::  +:once+ that accepts instead of fails. Every baseline is
+    #           compared; a screenshot that differs is kept as the new
+    #           baseline and never raises -- the bulk-accept verb, for the
     #           redesign that changed forty screenshots at once. Refused
     #           under CI (see {SnapDiff::ScreenshotMatcher}).
     #

lib/snap_diff/dsl.rb

@@ -76,7 +76,8 @@ def screenshot_group(name)
     #   so a preceding `preload_all_images` still costs its browser
     #   round-trips while the block costs nothing. Readiness work belongs
     #   inside the same switch as the capture it serves.
-    # @return [Boolean] True if the screenshot was successfully captured and processed.
+    # @return [Boolean] True if the screenshot was compared against a baseline; false when it had none (it is
+    #   captured and registered as a `:new` assertion) or screenshots are disabled.
     # @raise [SnapDiff::ExpectationNotMet] If comparison fails and immediate validation is enabled.
     # @raise [SnapDiff::UnstableImage] If the image comparison is unstable.
     # @raise [SnapDiff::WindowSizeMismatchError] If the window size doesn't match expectations.
@@ -94,15 +95,11 @@ def assert_matches_screenshot(name, skip_stack_frames: 0, **options)
         .new(full_name, options)
         .build_screenshot_assertion(skip_stack_frames: skip_stack_frames + 1)
 
-      return false unless assertion
-
-      # Registered either way: the end-of-run tally and the reporters see
-      # every finished assertion, immediate ones included. An immediate one
-      # validates here and is a no-op when the registry verifies at teardown.
-      SnapDiff.session.add_assertion(assertion)
+      # Already registered by the matcher. An immediate one validates here
+      # and is a no-op when the registry verifies at teardown.
       assertion.validate! unless options.fetch(:delayed, SnapDiff.config.delayed)
 
-      true
+      !assertion.compare.nil?
     end
 
     # Convenience wrapper around {#assert_matches_screenshot} and {#capture_screenshot}.

lib/snap_diff/reporters/ai_simple.rb

@@ -44,10 +44,9 @@ def record(assertions)
         return unless @backend
 
         assertions.each do |a|
-          difference = a.compare&.difference
-          next unless difference&.different?
+          next unless a.status == :changed
 
-          analyze_once(a.name, difference)
+          analyze_once(a.name, a.compare.difference)
         end
       end