Repository navigation
Real-server test coverage: 20 client operations have no test, session flows first #380
Description
Measured on master 2f6e41e. For each public client operation (a public …Async method on a *Client class), the measurement checked whether a test calls it. A test counts as real-server when it is a *LiveTests class or takes a pinned server fixture. The match is by name (.Method(), so an overload can be missed.
- 144 operations. 92 (63%) are called by a real-server test; 124 (86%) by any test.
- No test at all (20):
- Sessions:
AddSyntheticMessageAsync,CommitRevertAsync,StageRevertAsync,MoveToolsToBackgroundAsync,RunCommandAsync,RunShellCommandAsync,SwitchAgentAsync,SwitchModelAsync - Integrations:
BeginCommandConnectionAsync,CancelCommandConnectionAsync,CancelOAuthConnectionAsync,CompleteOAuthConnectionAsync,ConnectWithExternalCredentialsAsync,GetOAuthStatusAsync - Experimental:
ActivateSessionSkillAsync,AddIntegrationWellknownAsync - Others:
Credentials.RemoveCredentialAsync,McpServers.GetResourceCatalogAsync,OpenCodeClient.ReloadLocationsAsync,Websearch.GetProvidersAsync
- Sessions:
- Tested, but never against the real server:
- Sessions:
ClearRevertAsync,CompactAsync,ForkAsync,GenerateTextAsync,GetContextAsync,GetMessageAsync,MarkViewedAsync,SetEnvironmentAsync - Credentials, McpServers and Websearch: none of their operations
- Integrations: all but one operation
- Shells:
CreateShellAsync,GetOutputAsync,ListShellsAsync - Vcs:
GetVcsAsync,InitializeRepositoryAsync - Projects:
UpdateProjectAsync - Experimental: ten operations
- Sessions:
Scope (maintainer, 2026-10-16: low-hanging fruit only). Add real-server tests where the existing simulated-drive fixture already reaches the operation, with no new fixture, provider or network. The session operations come first, because they are the flows consumers use. For every operation that needs an external provider, OAuth, network (web search) or new infrastructure, record the reason here and leave it out.
Done when each listed session operation has a real-server test or a stated reason, and this issue lists the remaining operations with their reasons.
Activity
Blind-Striker commentedon Oct 16, 2026
Plan and status. I reviewed every listed operation against the pinned upstream (v2.0.29,
042d0832) and the existing simulated-drive fixture. 34 new real-server tests are planned, in four PRs, session operations first:A test counts only if it fails when its operation does nothing or goes to the wrong route. No empty-result, unknown-id or status-only test is added as coverage.
Left out of this low-hanging-fruit sweep:
Sessions.MoveToolsToBackgroundAsync: the fixture's echo tool is not a blocked job, and no deterministic foreground job, shell or subagent barrier exists. Adding one is new infrastructure.Integrations.BeginCommandConnectionAsync,CancelCommandConnectionAsync,GetCommandStatusAsync: no command-auth method or process is seeded, so no owned attempt exists to start, cancel or inspect.Integrations.BeginOAuthConnectionAsync,CancelOAuthConnectionAsync,CompleteOAuthConnectionAsync,GetOAuthStatusAsync: they need an OAuth authorization endpoint and exchange that the fixture does not have. Cancelling an unknown id only tests the no-op arm.Integrations.ConnectWithExternalCredentialsAsync: the simulated integration has no external credential source. Using a real one, or registering a new one, is out of scope.Experimental.ActivateSessionSkillAsync: no skill source is seeded, and project discovery is off in this fixture.Experimental.AddIntegrationWellknownAsync: URL discovery needs the network or a new endpoint.Experimental.ConnectMcpServerAsync,DisconnectMcpServerAsync: no connectable MCP process or endpoint exists. Disconnecting an already-disabled entry cannot tell a no-op apart.McpServers.GetResourceCatalogAsync: a non-empty catalog needs an MCP server, and an empty result would pass for a no-op.Websearch.GetProvidersAsync,QueryAsync: no search provider is registered. A useful test needs a provider or the network.Shells.ListShellsAsync: only running commands are listed, and the fixture has no deterministic held-shell arrangement. Sleeps, retries and new gates are out of scope.OpenCodeClient.ReloadLocationsAsync: a reload rebuilds every cached location, including directories that earlier tests have deleted, so a success test would depend on test order or need a separate clean fixture.Corrections to the measurement above:
Vcs.GetVcsAsyncalready runs against the real server, throughLiveReadiness.GitAsyncinVcsClientLiveTests.IntegrationClient.GetAsyncdoes not inherit live coverage from other clients'GetAsynccalls.Blind-Striker commentedon Oct 16, 2026
Progress. #387 merged (
295f5d35): nine session operations now have real-server tests. They areAddSyntheticMessageAsync,RunCommandAsync,RunShellCommandAsync,SwitchAgentAsync,SwitchModelAsync,MarkViewedAsync,SetEnvironmentAsync,ForkAsyncandGetMessageAsync. Each was shown red against the operation made a no-op, then green. Known limit: the fixture declares one model, soSwitchModelAsynccannot tell the requested model from a fallback to the default.Next: PR 2 is in progress:
StageRevertAsync,CommitRevertAsync,ClearRevertAsync,CompactAsync,GenerateTextAsyncandGetContextAsync. With it, every listed session operation has a real-server test, exceptMoveToolsToBackgroundAsync(out, with its reason above). PRs 3 and 4 follow.Blind-Striker commentedon Oct 17, 2026
PR 3 (#423) adds nine real-server tests: shells (create, output paging), VCS (re-initialisation refusal), projects (display-name update), and the experimental MCP add/remove and session-instruction list/remove/set.
Moved to the out-list: E02,
ExperimentalClient.GenerateTextAsync. A no-op mutant leaves the simulated drive's invocation waiter pending before the text oracle is reached. A test that fails at its oracle, rather than at a timeout, therefore needs per-call cancellation in that waiter. That is harness work beyond the short arrange this sweep allows (lead limit). The operation is reachable; it is left uncovered for that reason only.test: credentials, integrations and the MCP catalog run against the r…
Blind-Striker commentedon Oct 17, 2026
The real-server coverage sweep is complete: 33 new live tests in four PRs, all on the existing simulated-drive fixture. They add no fixture, provider, support type or dependency, and no SDK behaviour change.
The bar for every test. Each one fails at its oracle when its operation is made a no-op that returns a valid success shape, or sent to the wrong route. It fails by an assertion, never a timeout. The PRs record each mutation and its red run. No empty-result, unknown-id or status-only assertion counts as coverage. Credential tests use inert sentinel values and never read, print or compare a real credential.
Inventory corrections (they need no test):
Vcs.GetVcsAsyncalready runs live throughLiveReadiness.GitAsync.Not covered: the out-list. The following are reachable, but a no-op-proof test needs harness infrastructure this sweep was scoped not to build. Each is recorded, with the infrastructure that unlocks it, in #430:
ConnectMcpServerAsync,DisconnectMcpServerAsync,GetResourceCatalogAsync.BeginCommandConnectionAsync,CancelCommandConnectionAsync,GetCommandStatusAsync.BeginOAuthConnectionAsync,CancelOAuthConnectionAsync,CompleteOAuthConnectionAsync,GetOAuthStatusAsync.ConnectWithExternalCredentialsAsync.AddIntegrationWellknownAsync.Websearch.GetProvidersAsync,Websearch.QueryAsync.MoveToolsToBackgroundAsync,ShellsClient.ListShellsAsync.Experimental.GenerateTextAsync, dropped from PR 3.ActivateSessionSkillAsync.OpenCodeClient.ReloadLocationsAsync.None of these is untestable; each is a known gap with its prerequisite named. Closing #380 as completed. The remaining coverage is tracked in #430.