Skip to content

Flaky live test: GetConfigAsync_Should_Read_Back_The_Shell_UpdateConfig_Wrote never sees config.updated (Windows net48) #440

Closed

Description

@Blind-Striker

ConfigClientLiveTests.GetConfigAsync_Should_Read_Back_The_Shell_UpdateConfig_Wrote failed once on Windows, net48. It ejected #439 from the merge queue. #439 changes only ModelMaterializationMatrixTests, a clone baseline and one quality-gates.md sentence, so it cannot reach this test.

Evidence: merge-group run 39026886983 (gh-readonly-queue/master/pr-439-31cf8a81), job "Build & Test (Windows)", 2026-10-17 11:05Z:

failed GetConfigAsync_Should_Read_Back_The_Shell_UpdateConfig_Wrote (1m 00s 229ms)
TimeoutException: No event satisfied 'config.updated for the written shell' before the wait ended.
Events: 'server.connected', 'skill.updated', 'skill.updated', 'skill.updated', 'skill.updated'.
  at OpenCode.Sdk.Tests.Support.SessionEventProbe.WaitForAsync

UpdateConfigAsync returned 204 (the assertion before the wait passed). After that, the probe saw four skill.updated events and no config.updated for the whole 60-second barrier (tests/OpenCode.Sdk.Tests/Config/ConfigClientLiveTests.cs:123, wait at :158). In the same run, the same test passed on Windows net10.0 and net11.0, and on Linux and macOS. The job's summary attributes the one failure to OpenCode.Sdk.Tests.exe (net48|x64) (+2799/x1); the failure line itself is printed among net11.0's interleaved output. It also passed on all three OS legs in #438's and #439's PR runs.

Open question for the owner: did the server not publish config.updated after the write that one time? Or was it published before the subscription or the probe could see it, or on a bus the probe doesn't watch? The steady skill.updated stream suggests the instance reloaded after the write.

Not fixed by #439; recorded under the findings rule in docs/engineering/workflow.md.

Activity

  1. changed the title [-]Flaky live test: GetConfigAsync_Should_Read_Back_The_Shell_UpdateConfig_Wrote never sees config.updated (Windows net11.0)[/-] [+]Flaky live test: GetConfigAsync_Should_Read_Back_The_Shell_UpdateConfig_Wrote never sees config.updated (Windows net48)[/+] on Oct 17, 2026
  2. Blind-Striker commented on Oct 17, 2026

    @Blind-Striker
    CollaboratorAuthor

    Root cause: cross-test config residue on the shared server (test isolation, not product)

    Measured from the TRX of merge-group run 39026886983, plus three green master runs (39022428987, 39022831816, 39025318239):

    • On net48, UpdateConfigAsync_Should_Persist_And_Clear_The_Shell always runs before GetConfigAsync_Should_Read_Back_The_Shell_UpdateConfig_Wrote. Both share one SimulatedDriveServerFixture, which ParallelConstraintKeys.ServerProcess serializes.
    • In the red run, Persist took 4.0 s (11:04:19.74–23.73) and Read_Back started 130 ms after it ended. In green runs Persist finishes in under 0.5 s.
    • On net10.0 and net11.0, Read_Back runs first, and those legs pass. Neither the SSE path nor net48 is at fault: net48 is just the only leg where Read_Back follows Persist, and this time the runner was slow.

    Mechanism, read in upstream packages/core/src/config.ts at the pin (lines 316-328 and 380-401):

    • experimental.config.update writes the global file and only requests a reload. The request goes through a sliding PubSub of capacity 1 and a 100 ms debounce, and reloadLock serializes the reloads.
    • A reload publishes config.updated only when the loaded entries change (isDeepStrictEqual).
    • Persist writes the shell, then null, and checks only the file, never the loaded state. So the test ends with reloads still in flight.
    • When its set-reload is slow, the loaded state becomes the shell value and config.updated is published before Read_Back subscribes. Its skill.updated reactions arrive later: these are the four events Read_Back's probe saw.
    • The reload left pending by Persist's clear then reads the file only after Read_Back has written the same shell. It sees no change and publishes nothing, and neither does Read_Back's own reload. The result is a 60 s timeout.

    Upstream behaves as designed. The defect is that one test leaves the shared server's configuration unsettled, while the other assumes a settled baseline.

    No other test writes this server's global configuration: ConfigClientLiveTests holds the only UpdateConfigAsync calls, and no test writes into the run root's config directory at test time.

    Fix plan:

    1. A deterministic red repro first: write the shell, wait for config.updated, then write null and the shell back to back within 100 ms. Expected: no config.updated.
    2. Then fold Persist's file assertions into Read_Back, so one test owns the whole write, settle, clear, settle cycle.

    No retry and no wider bound.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions