Repository navigation
chore: validate dedicated Windows hardware runners - #2024
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
|
Codex review: needs maintainer review before merge. What this changesAdds an opt-in workflow that builds and tests on two dedicated ARM64 runners and one x64 runner. Example: A maintainer pushes to ci/hardware-validation.
Review scores
ProductKind: Maintenance · Worth it: Yes · Fix scope: Complete Merge readiness✅ Ready for maintainer review Keep open: this maintainer-authored PR adds useful, distinct hardware validation absent from main; no definite patch defect was found, but two hardware test failures remain unattributed. Priority: P2 Before mergeNone. FindingsNone. Tests
Agent review detailsHow this fits togetherGitHub Actions routes an explicit dispatch or dedicated-branch push to persistent Windows runners, producing build results and uploaded TRX reports. flowchart TD
A[Manual dispatch or dedicated branch push] --> B[Canonical repository gate]
B --> C[Three Windows hardware runners]
C --> D[Native architecture and prerequisite checks]
D --> E[Repository build]
E --> F[Shared and tray tests]
F --> G[TRX admission and artifacts]
Technical reviewBest possible solution: Retain the focused opt-in workflow and have the CI owner classify the two hardware test failures before landing. Do we have a high-confidence way to reproduce the issue? This is infrastructure work rather than a reported application bug; the exact-head hardware run confirms that all three runner jobs execute. Is this the best way to solve the issue? A separate opt-in matrix is a proportionate solution that reuses existing build and test owners while preserving ordinary PR CI. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 4aee38a1bfaa. TestingProof path: shipped entry point. SecurityNone. EvidenceWhat I checked:
Preflight sourceSource unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified. LabelsLabel changes:
Label justifications:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment Reviewed November 5, 2026, 5:11 PM ET / 22:11 UTC. |
What Problem This Solves
Dedicated Windows hardware needs an opt-in validation workflow with native builds and nonempty test results.
User Impact
Maintainers can explicitly validate both RTX ARM64 devices and the Azure x64 host without routing arbitrary pull requests onto persistent machines. Socket and TLS failure observations retain their typed cause even when an outer message includes a port such as 50321.
Why This Change Was Made
Adds an opt-in three-host workflow: native architecture and prerequisite checks, full repository build, and shared/tray tests with nonempty TRX admission. Manual dispatch is supported after merge; pushes to the dedicated
ci/hardware-validationbranch provide pre-merge execution. Existing CI and release lanes are unchanged. The branch now consumes current main, including the transport-classification improvements and the native UI Automation repair in #2050. The merge keeps main’s classifier unchanged and retains the endpoint-digit regression; no duplicate product fix is added.Evidence
Current candidate:
3740ef01b65498bbc0303d1e54d00caae3e8c38aincorporates #2050, which fixes the disabled-action native UIA HRESULT and reproduces the stale COM-error mechanism. The sole main-branch conflict was resolved by preserving main’s classifier exactly. GitHub now reports the branch mergeable. Fresh integrated-head CI passed, including CI Gate, on this exact head.The UI failure was separate from Azure’s runner login issue. Azure and both RTX hosts already passed at the earlier head, as recorded below. No tests were skipped or weakened to address either failure.
Earlier hardware and CI attempts (preserved evidence at 4c7bb57)
All three hardware runners passed full native builds and shared/tray suites at earlier head
4c7bb57c. The hosted UI-test failure motivated the follow-up above. Exact-head hardware run 42121609752 passed both RTX jobs: each full build, 4,930 shared tests passed (24 skipped), and 5,062 tray tests passed (0 skipped). Azure built successfully and passed 4,928 shared tests (25 skipped), but one unrelated config-request fixture watchdog timed out; tray tests were not reached. All three hosts passed the transport-classification regression. Attempt 2 cleared the watchdog (4,929 shared passed, 25 skipped), then failed one tray test at CurrentUser DPAPI encryption (5,061 passed, 1 failed). The Azure runner used passwordless S4U login. A credential-backed Password task at the same Limited run level passed a synthetic DPAPI round-trip and the exact failing tray test (1 passed, 0 failed, 0 skipped). The idle runner was switched to that login configuration without rebooting or changing the SSH relay. However, Azure-only attempt 3 failed the same DPAPI test: build passed, shared 4,929 passed / 0 failed / 25 skipped, tray 5,061 passed / 1 failed / 0 skipped. The focused pass did not establish full runner-context recovery: live inspection found the old S4U listener still running after Stop-ScheduledTask. The replacement Password task had exited with a runner-session conflict, so attempt 3 still used the old login. After verifying the runner was idle and stopping only that orphaned listener, the replacement task remained Running and its actual CI worker/testhost shared the new logon session. Azure attempt 4 passed the full build, 4,929 shared tests (25 skipped) and 5,062 tray tests (0 skipped), including the exact formerly failing encryption test. Artifact upload and isolated-settings cleanup passed. No source/test changes, reboot, elevation, or relay changes were needed for this repair. The regular PR run 42120132261 attempt 2 completed with a different unchanged UI-test failure:OnboardingAiPageTests.CurrentModel_NativeUiaRespectsActionAvailability(disabled: True)expectedCOMExceptionbut receivedElementNotAvailableExceptionat line 159 (641 passed, 1 failed, 7 skipped). Attempt 1 test-host crash is retained. No further UI retry or test suppression performed; that earlier run’s CI Gate remained failed. The deterministic regression failed both new cases on the previous source, then the patched handshake-observation class passed 12/12, including real loopback connection refusal. Independent workflow and source/test review completed with no outstanding findings.Earlier hardware run 42116050771 passed full builds on all three hosts and all executed tests on both RTX hosts. The next run reproduced the classifier defect and exposed separate Azure process-fixture timeouts while post-install native-image compilation and Defender saturated its two CPUs. Original failures are retained; no tests or deadlines were weakened.
Change Type
Scope
winnodeRequired proof pools
windows-winui-interactive: native UI Automation invocation and disabled-action rejection on an isolated desktop; not a claim of complete accessibility or visual qualification.windows-11-arm64: native build and test execution on both RTX devices. This does not qualify all ARM64 UI/package behavior.Validation
./build.ps1: passed. Shareddotnet test tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj -r win-arm64 -p:AppendRuntimeIdentifierToOutputPath=false --no-restore: 4,869 passed / 0 failed / 24 skipped. Same command forOpenClaw.Tray.Tests: 5,237 passed / 0 failed / 0 skipped.dotnet test tests/OpenClaw.Tray.UITests -r win-arm64 --filter "FullyQualifiedName~OnboardingAiPageTests&Category!=NativeOnboardingProof": 106 passed / 0 failed / 0 skipped, including enabled invocation and both disabled/unavailable native UIA cases. TRX results, full build transcript, exit receipt, and private-desktop receipts retained. The separate NativeOnboardingProof category is outside this selected suite, not a newly added exclusion../build.ps1passed; shared 4,930 passed / 24 skipped, tray 5,062 passed / 0 skipped, zero failures. This is old head plus the preserved duplicate patch, not integrated-head evidence.actionlint .github/workflows/hardware-validation.ymlandgit -c core.whitespace=cr-at-eol diff origin/main --check: passed.Earlier-head evidence below is retained, not relabeled as updated-head CI:
Azure login repair proof: synthetic CurrentUser DPAPI protect/unprotect passed under a credential-backed Limited scheduled task.
dotnet test tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj -r win-x64 -p:AppendRuntimeIdentifierToOutputPath=false --no-build --no-restore --filter FullyQualifiedName~SettingsPersistenceNotificationTests.TypedStoreRollbackAndExplicitReloadPreserveExternalFieldsAndProtectedSecrets: 1 passed, 0 failed, 0 skipped, exact head4c7bb57c.PASS:
actionlint .github/workflows/hardware-validation.ymlPASS:
git diff --checkPASS:
pwsh -NoProfile -File scripts/test-ci-workflow-contract.ps1RED control: detached
db205eacplus the new regression,dotnet test tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj -r win-arm64 -p:AppendRuntimeIdentifierToOutputPath=false --filter FullyQualifiedName~TransportFailure_TypedCauseTakesPrecedenceOverEndpointDigits: 2 failed, 0 passed for the intended incorrectServerclassification.GREEN focused proof: matching source/test bytes from
4c7bb57c, same native test command with--filter FullyQualifiedName~GatewayHandshakeObservationTests: 12 passed, 0 failed, 0 skipped.Earlier full proof at
9333aa23: both RTX hosts passed./build.ps1, shared tests 4,928 passed / 24 skipped, and tray tests 5,062 passed / 0 skipped. Azure full build passed; its shared-test failure is retained above. These are not current-head full-run claims.Earlier-head (
4c7bb57c) full./build.ps1: passed on all three hosts. That head's shared/tray suites passed on both RTX hosts with the counts above. Azure attempt 1 failed the request-timeout fixture; attempt 2 passed shared tests but failed DPAPI in one tray test. Credential-backed Limited login passed the focused test, but full Azure attempt 3 still failed the same DPAPI test (shared 4,929 passed / 25 skipped; tray 5,061 passed / 1 failed). Azure attempt 4 subsequently passed: 4,929 shared tests / 25 skipped and 5,062 tray tests / 0 skipped after replacing the surviving S4U listener. Original attempts and TRX artifacts are retained.Real Behavior Proof
Current-head native proof passed on an isolated RTX1 ARM64 desktop (SSH static host, Interactive/Limited task, no shared input-desktop switch). All 106 selected onboarding cases passed, including the seeded stale-IErrorInfo disabled-action regression and positive invocation control. Hosted CI and CI Gate passed on the same head. Earlier hardware validation at
4c7bb57cis green on all three hosts after the Azure orphaned-listener repair. Main’s #2050 independently reproduced the stale-error failure mechanism; native integration proof now passes; hosted UI, functional, accessibility, and WSL E2E jobs passed.3740ef01b65498bbc0303d1e54d00caae3e8c38a(native build/shared/tray/onboarding passed; hosted CI passed); earlier three-host hardware proof4c7bb57cbe973734d22c9f1856a735613f190d85.Security Impact
Execution is limited to explicit dispatch or a maintainer-pushed opt-in branch in the canonical repository. No pull-request trigger, signing secret, or write token. Checkout credentials are not persisted. Tests use run-specific tray settings; this is not machine-level isolation.
Yes, explain the risk and mitigation: only explicit dispatch or the canonical opt-in branch runs this workflow; isolated test settings and scoped cleanup do not claim machine-level isolation.Compatibility and Migration
Additive opt-in workflow; existing CI is unchanged. Hardware runners require the custom labels declared in actionlint configuration. Azure must remain available for a three-host run.
Review Conversations