Repository navigation
test: split the reviewer usage-limit test per case and unskip it - #537
Conversation
…th-its-reset-and-records-no-round.md Quest-Claim: 7ac2eaeb9a89df07
All four cases ran two full reviews each inside one 5s test, which outran its timeout on loaded merge-group runners. One test per case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed head 60848cb against base 1c3cf6a.
No actionable findings. Splitting the four host/reset combinations into separate fakes.for tests preserves the stderr, exit-code, empty-stdout and subsequent successful-review assertions while giving each pair of review runs its own timeout. The existing fakes wrapper retains the Windows exclusion, each case gets a fresh setup, and afterEach restores stubbed environment variables. Removing the now-empty flaky epic, completed leaf and parent link is consistent with the completed work.
Direction: proceed through normal CI/merge validation; use loaded merge-group runs to confirm the timeout improvement rather than treating the split alone as proof that the flake is eliminated.
Verification limits: GitHub-only static inspection of the diff, test fixtures and flaky-quest handling. I did not run the tests, reproduce runner load, or independently verify the reported timings.
(Written by review (OpenAI))
|
quest land watch: the merge queue removed this pull request at 60848cb because its The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.
…oup size are both one in main's ruleset.
�[36;1m# Build concurrency and group size are both one in main's ruleset.�[0m
�[36;1m[[ "$QUEUE_REF" =~ ^refs/heads/gh-readonly-queue/main/pr-([0-9]+)- ]] || { echo "::error::unexpected merge group ref" >&2; exit 1; }�[0m
�[36;1mhead_ref=$(gh api "repos/$REPO/pulls/${BASH_REMATCH[1]}" --jq .head.ref)�[0m
�[36;1m./quest-base/bin/quest --root "$GITHUB_WORKSPACE" check --base "$BASE" --branch "$head_ref"�[0m
shell: /nix/store/1mv3qz005gkbalxfghk4y3s8ayv4d2dl-bash-interactive-5.3p15/bin/bash -e {0}
env:
BASE: 6b6ce221323fda9350f3dbc84232bd5f5867a3d8
GH_TOKEN: ***
REPO: moq-dev/quest
QUEUE_REF: refs/heads/gh-readonly-queue/main/pr-537-6b6ce221323fda9350f3dbc84232bd5f5867a3d8
##[endgroup]
quest: 77 documents ok
##[group]Run nix develop --command just install
�[36;1mnix develop --command just install�[0m
shell: /nix/store/1mv3qz005gkbalxfghk4y3s8ayv4d2dl-bash-interactive-5.3p15/bin/bash -e {0}
env:
BASE: 6b6ce221323fda9350f3dbc84232bd5f5867a3d8
##[endgroup]
bun install --frozen-lockfile
bun install v1.4.4 (38afee6c8)
+ @types/mdast@4.0.4
+ @types/node@26.6.6
+ prettier@3.9.10
+ typescript@7.0.2
+ vitest@5.0.5
+ @opencode/client@2.2.1
+ commander@15.0.0
+ mdast-util-from-markdown@2.1.0
+ mdast-util-gfm-strikethrough@2.0.1
+ mdast-util-gfm-table@2.0.0
+ micromark-extension-gfm-strikethrough@2.1.0
+ micromark-extension-gfm-table@2.1.2
187 packages installed [1307.00ms]
##[group]Run nix develop --command just check
�[36;1mnix develop --command just check�[0m
shell: /nix/store/1mv3qz005gkbalxfghk4y3s8ayv4d2dl-bash-interactive-5.3p15/bin/bash -e {0}
env:
BASE: 6b6ce221323fda9350f3dbc84232bd5f5867a3d8
##[endgroup]
bun run --silent quest gate just _check
bun run check
$ tsc && tsc -p src/core && prettier --check .
Checking formatting...
[�[33mwarn�[39m] test/review.test.ts
[�[33mwarn�[39m] Code style issues found in the above file. Run Prettier with --write to fix.
error: script "check" exited with code 1
error: recipe `_check` failed on line 19 with exit code 1
error: recipe `check` failed on line 16 with exit code 1
##[error]Process completed with exit code 1.
Post job cleanup.
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git version
git version 2.55.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/d74ebe12-1a2d-4997-a96d-a9ef13bdf864' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/quest/quest
Removing SSH command configuration
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Post job cleanup.
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git version
git version 2.55.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/c42c89f8-95b0-479b-870b-2fa79b9def3b' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/quest/quest
Removing SSH command configuration
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/lg46w7hrjx7kylsh6645c9l975i8h43b-git-2.55.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
A job completed hook has been configured by the self-hosted runner administrator
##[group]Run '/nix/store/gx2v6q84b0w43bafm0whkdx93qyxbpps-ci-cache-gc.sh/bin/ci-cache-gc.sh'
shell: /nix/store/1mv3qz005gkbalxfghk4y3s8ayv4d2dl-bash-interactive-5.3p15/bin/bash --noprofile --norc -e -o pipefail {0}
##[endgroup]
evicted 0 objects and 0 action results (0 B logical); 45.9 GiB logical remain
Cleaning up orphan processes
Current runner version: '2.338.2'
##[group]Runner Image Provisioner
Hosted Compute Agent
Version: 20261028.624
Commit: 54898b74c23fde5556c9b3c3a2290865feef0dfe
Build Date: 2026-10-28T21:51:42Z
Worker ID: {289274b9-b408-4376-ae0d-16d31d6f8f14}
Region: eastus2
Cloud: Azure
##[endgroup]
##[group]Operating System
Ubuntu
24.04.5
LTS
##[endgroup]
##[group]Runner Image
Image: ubuntu-24.04
Version: 20261101.360.1
Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20261101.360/images/ubuntu/Ubuntu2404-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20261101.360
##[endgroup]
##[group]GITHUB_TOKEN Permissions
Contents: read
Metadata: read
PullRequests: read
##[endgroup]
Secret source: Actions
Cache mode: write
Prepare workflow directory
Prepare all required actions
Complete job name: Queue
##[group]Run [[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }
�[36;1m[[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }�[0m
shell: /usr/bin/bash -e {0}
env:
RESULT: failure
##[endgroup]
##[error]the queue build ended failure
##[error]Process completed with exit code 1.
Cleaning up orphan processes
(written by quest land watch) |
|
quest land watch: armed auto-merge on this pull request at 2759882, so it joins the back of the merge queue once its required checks pass, with the land fixer's fix ( (written by quest land watch) |

Fixes the flaky test skipped in moq-dev/quest#533.
Root cause: one test ran four cases × two full
quest reviewruns (each spawning fake reviewer CLIs). That took ~2.4s alone and outran the 5000 ms timeout on loaded merge-group runners. Nowfakes.formakes one test per case (~0.5–0.6s each), with no timeout, retry, or sleep changes.Choices made without asking:
quest/m0/flakyepic and its link; the queue watcher recreates it when missing (seequest/m0/queue-watcher/flaky-quest.md).moq-dev/quest#517 edits other parts of
test/review.test.ts; expect a trivial merge.🤖 Generated with Claude Code
(Written by Claude Opus 5.5)
Review
mechanical fix by codex, gpt-6.1-sol, medium effort:
Merged base cleanly and formatted test/review.test.ts without behavior changes; ran just install, bunx --no-install prettier --write test/review.test.ts, just check, just test, and nix flake check --option substituters ''.(head 2759882)Rewritten by
quest reviewafter each round.01a1b764-ac59-7043-a598-ca09d6554b69, round 1The change preserves the assertions while giving each case its own timeout and the existing Windows skip, and consistently removes the completed quest; tests were not run in the read-only environment.