fix(test): v1.3.0's release gate failed on Linux — a test helper counted event-loop turns - #97
Merged
Merged
Conversation
…ut on the Linux release gate v1.3.0's release gate failed in authRetryCallers.test.js: "never happened: the renewal", in the first inline-completion case that polls. The suite had passed on every machine it had been run on, all of them Macs. until() gave a condition 400 turns of the event loop to come true. Inline completion waits out its debounce on a real timer, and a timer is at least a millisecond even when the setting is 0. How many turns fit into that millisecond is the machine's business: macOS a 0 ms timer fires after ~70 turns; 400 turns take ~5 ms Linux it fires after ~1,500 turns; 400 turns take ~0.3 ms So on the runner the count ran out before the timer had fired, and the request the case was waiting on had not been sent yet. Nothing is wrong with the code under test. The three cases that poll across the debounce are the ones exposed; which of them fails first varies from run to run. until() now waits by the clock: up to two seconds, the patience within() already has. A condition that never comes true still fails, with the same message. settle() stays a count of turns, and says what that is good for: only what no timer stands in the way of. Both places that use it are that. Reproduced before fixing, without the runner: in a Linux container the tag fails 3 runs of 3, and on a Mac with every timer made 15 ms slower it fails with the runner's exact message. With the fix: 60 of 60 in the container, all 48 suites there three times over, and all 48 on a Mac with and without the slowed timers. No other suite counts turns across a timer; the slowed-timer run would have shown one. Not run on the GitHub runner itself yet, and the container has Node 18 where the runner has 24. The next commit makes the pull request do that run.
…se gate's own script The suites ran in CI at one moment only: when a tag was pushed. A pull request got CodeQL and nothing else, so the first Linux run of any suite was the release gate, after the merge. That is how v1.3.0 was tagged on a suite that could not pass there (see the previous commit). The gate moves, unchanged, into scripts/test-extensions.sh: discover every extensions/*/test/*.test.js, stop at the first failing suite, and fail if the glob matches nothing. release.yml calls it where the inline loop was, and a new workflow, test.yml, calls it on every pull request and on pushes to develop — same runner image, same Node. One script, so the two cannot drift apart. test.yml asks for read access only and needs no secrets. A newer push to the same branch cancels the run before it. Checked: the script passes all 48 suites on a Mac and in a Linux container under the runner's shell flags; it exits with a failing suite's own code without running the next; it exits 1 when nothing matches. actionlint is clean on both workflow files. release.yml's gate step is the one thing here not exercised until a tag is pushed — but it is now a single line that test.yml runs on this very pull request.
…nner's own The comment on until() gave a figure for "the Linux runner the release gate uses". The figure was measured in a Linux container; on the runner itself only the outcome is known, that 400 turns ran out. Say Linux, where the gate runs.
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The timing fix and centralized CI gate are consistent, bounded, and correctly wired.
Review effort: Balanced
Findings: None
What changed in this PR
Fixes a Linux-only timing failure and moves extension tests into pull-request CI.
Changes:
- Replaces event-loop-turn polling with a two-second deadline.
- Centralizes extension test discovery in a reusable script.
- Runs the same test gate for releases, pull requests, and
develop.
| File | Description |
|---|---|
extensions/levelcode-ai/test/authRetryCallers.test.js |
Makes asynchronous waits machine-independent. |
scripts/test-extensions.sh |
Adds the shared extension test runner. |
.github/workflows/test.yml |
Adds pull-request and develop test coverage. |
.github/workflows/release.yml |
Reuses the shared test runner. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What failed
git push --tagsfor v1.3.0 started the release workflow, and its test gate went red before any build ran (run):The suite came in with #96 and passed everywhere it had been run. Every one of those machines was a Mac.
Why
Nothing is wrong with the code under test. The test's
until()gave a condition 400 turns of the event loop to come true. Inline completion waits out its debounce on a real timer, and a timer is at least a millisecond even when the setting is 0. How many turns fit in a millisecond depends on the machine:On the runner the count ran out before the timer had fired, so the request the case was waiting for had not been sent yet.
The figures are from a Mac and from a Linux container on the same machine. The runner itself was not measured. What is known of it is the outcome in the log: its 400 turns ran out.
The fix (first commit)
until()waits by the clock: up to two seconds, the patiencewithin()already has. A condition that never comes true still fails, with the same message.settle()stays a count of turns and now says what that is good for. Both places that use it have no timer in the way.One file, test-only:
extensions/levelcode-ai/test/authRetryCallers.test.js.So it is found on the pull request next time (second commit)
The suites ran in CI at one moment only: when a tag was pushed. A pull request got CodeQL and nothing else, so the first Linux run of any suite was the release gate, after the merge.
scripts/test-extensions.sh.release.ymlcalls the script where the inline loop was.test.yml, calls it on every pull request and on pushes todevelop. Same runner image, same Node. It asks for read access only and needs no secrets.This commit is separable. The first commit alone turns the gate green. Drop the second if you would rather not have a pull-request check.
Verification
never happened.actionlintis clean on both workflow files.Extension unit testscheck on this pull request is that run. Its first run passed all 48 suites on the gate's own image and Node,ubuntu-24.04with Node 24.21.0, including the case that failed at the tag.Not exercised until a tag is pushed:
release.yml's gate step itself. It is now one line, calling the script this pull request runs.After merging
v1.3.0 points at the commit before this fix, and its run published nothing: the build and the draft release were skipped. Move the tag to the merged commit and push it again: