Skip to content

test(gmail): make TestFetchMessageDetails_NoRetryOnError deterministic - #1256

Closed
malob wants to merge 1 commit into
openclaw:mainfrom
malob:test/fetch-details-no-retry-deterministic
Closed

malob wants to merge 1 commit into
openclaw:mainfrom
malob:test/fetch-details-no-retry-deterministic

Conversation

@malob

@malob malob commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Since #1253 cancels the remaining detail fetches after the first failure, this test races: whether m2's request goes out before m1's error cancels it depends on scheduling, while the test still expected exactly 2 calls. Observed in #1255's test and minimum-go jobs.

Local reproduction results using go test ./internal/cmd/ -run 'TestFetchMessageDetails_NoRetryOnError$' -count=1000 -cpu=1:

Commit Failures
3ba47bd (before #1253) 0 / 1000
60ae623 (#1253) 74 / 1000
99a9d58 (this branch) 0 / 1000 (also 0 / 1000 at -cpu=2 and -cpu=8)

Changes

  • Count requests per message: m1, the failing one, must be requested exactly once (the no-retry check the test exists for), and m2 at most once, since the cancellation may or may not reach it first. The test's client bypasses the production retry transport, so this checks fetchMessageDetails itself.

Test-only; no behavior change, so no changelog entry.

🤖 Generated with Claude Code

Since openclaw#1253 cancels the remaining detail fetches after the first failure,
whether m2's request goes out before m1's error cancels it is a race, so
the test's exact-2-calls assertion failed intermittently (74 of 1000 runs
with -cpu=1). Count requests per message instead: m1, the failing one,
exactly once (the no-retry check), and m2 at most once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@clawsweeper

clawsweeper Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-environment Preliminary submission classification; full source review pending. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-environment Preliminary submission classification; full source review pending. labels Oct 23, 2026
@clawsweeper

clawsweeper Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

What this changes

This PR replaces an exact total-request assertion with per-message counts in the Gmail detail-fetch test.

Example: m1 returns HTTP 500 and cancellation prevents m2 from reaching the test server.

  • Before: The test fails because it expects two requests but observes one.
  • After: The test accepts one request for m1 and zero for m2 while still rejecting retries.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The focused patch is correct and supported by exact-head CI, but its helper-test evidence does not satisfy the explicitly required shipped-entry-point proof gate.
Proof confidence 🧂 unranked krab (1/6) Real behavior proof is necessary before merge. See Before merge.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Test only · Worth it: Yes · Fix scope: Complete
User problem: Contributors can receive failing CI results from a valid cancellation schedule.
Reason: The assertions preserve the test's original no-retry purpose and respect the cancellation behavior explicitly established by the merged maintainer contribution.

Regression provenance

Regression provenance — verified: source commit 60ae623da5b5 by Peter Steinberger; canonical PR #1253 (blame-to-merge-commit).

Merge readiness

⛔ Blocked before merge - 1 item remains

Keep this PR open: current main still contains the scheduling-dependent assertion, and this focused change preserves the no-retry check while accommodating intentional cancellation. No actionable code defect was found.

Priority: P3
Reviewed head: 99a9d589b9df25bbf3b8582fc04e8d23054fbc1c

Before merge

  • Add real behavior proof - The supplied evidence exercises fetchMessageDetails through its test HTTP server and reports repeated-run success, but provides no shipped CLI fault trace; the four inspected downloaded images show unrelated Gmail quote rendering. Under the stated external-PR gate, the contributor must supply a redacted Gmail-search CLI trace showing first-error cancellation, or obtain an explicit proof override for this test-only change. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.

Findings

None.

Tests

  • Missing end-to-end proof: The supplied repeated-test results exercise the helper against httptest, rather than the shipped Gmail search entry point.
Agent review details

How this fits together

Gmail message search fetches message details concurrently through the Google API client and cancels outstanding requests after its first error; this test exercises that helper against a local HTTP server.

flowchart LR
A[Gmail message search] --> B[Concurrent detail fetcher]
B --> C[Google API client]
C --> D[Gmail API or test server]
D --> E[First request error]
E --> F[Cancel outstanding fetches]
F --> G[Return listing error]
Loading

Technical review

Best possible solution:

Land the focused per-message assertions while preserving production cancellation and the existing fan-out regression coverage.

Do we have a high-confidence way to reproduce the issue?

Yes: current-main cancellation can prevent m2 from reaching the server while the test requires both requests; the author's repeated-run results corroborate this path, and no target code was executed during review.

Is this the best way to solve the issue?

The patch is the smallest complete correction: count the failing message exactly once and permit the other message to be canceled, using the existing test server and atomic counters.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against f9052a583e87.

Provenance checked

  • No-retry test request accounting changes intended behavior with a stated reason (07a0691: The message-search hardening commit added an HTTP-error test to require no retry; its exact total predates shared cancellation.)
  • Cancellation after the first detail-fetch failure keeps the original intent (60ae623: Stop unnecessary pending requests once the listing has already failed, as stated in fix: correct Gmail retries, queries, and Drive output #1253.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

Likely related people:

  • Peter Steinberger: Raw commit 07a0691 adds internal/cmd/gmail_messages_test.go:34 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 07a0691d5415; files: internal/cmd/gmail_messages_test.go)

Review metrics

Metric Value Why it matters
Change distribution Production +0/-0; tests +10/-4 The small test-only change removes a false failure without adding production machinery.

Labels

Label changes:

  • add P3: This repairs a flaky regression test without changing runtime behavior.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Label justifications:

  • P3: This repairs a flaky regression test without changing runtime behavior.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Provide the scoped CLI fault trace described in the proof assessment, or obtain an explicit test-only proof override.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 23, 2026, 2:26 AM ET / 06:26 UTC.

@steipete

Copy link
Copy Markdown
Collaborator

Thanks for isolating the cancellation race and preserving the no-retry assertion. This is now superseded by #1249: commit a20b2e2 changed TestFetchMessageDetails_NoRetryOnError to request only the failing message and assert exactly one request, so the test no longer depends on whether cancellation reaches a sibling first.

Current main (563e30a) includes that deterministic fixture and has green Linux, minimum-Go, Windows, and macOS CI. Closing as superseded; thank you for the focused fix and reproduction.

@steipete steipete closed this Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants