Skip to content

chore: validate dedicated Windows hardware runners - #2024

Merged
RomneyDa merged 5 commits into
openclaw:mainfrom
RomneyDa:task/crabbox-ci-20261105
Nov 7, 2026
Merged

RomneyDa merged 5 commits into
openclaw:mainfrom
RomneyDa:task/crabbox-ci-20261105

Conversation

@RomneyDa

@RomneyDa RomneyDa commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

What Problem This Solves

Dedicated Windows hardware needs an opt-in validation workflow with native builds and nonempty test results.

User Impact

Maintainers can explicitly validate both RTX ARM64 devices and the Azure x64 host without routing arbitrary pull requests onto persistent machines. Socket and TLS failure observations retain their typed cause even when an outer message includes a port such as 50321.

Why This Change Was Made

Adds an opt-in three-host workflow: native architecture and prerequisite checks, full repository build, and shared/tray tests with nonempty TRX admission. Manual dispatch is supported after merge; pushes to the dedicated ci/hardware-validation branch provide pre-merge execution. Existing CI and release lanes are unchanged. The branch now consumes current main, including the transport-classification improvements and the native UI Automation repair in #2050. The merge keeps main’s classifier unchanged and retains the endpoint-digit regression; no duplicate product fix is added.

Evidence

Current candidate: 3740ef01b65498bbc0303d1e54d00caae3e8c38a incorporates #2050, which fixes the disabled-action native UIA HRESULT and reproduces the stale COM-error mechanism. The sole main-branch conflict was resolved by preserving main’s classifier exactly. GitHub now reports the branch mergeable. Fresh integrated-head CI passed, including CI Gate, on this exact head.

The UI failure was separate from Azure’s runner login issue. Azure and both RTX hosts already passed at the earlier head, as recorded below. No tests were skipped or weakened to address either failure.

Earlier hardware and CI attempts (preserved evidence at 4c7bb57)

All three hardware runners passed full native builds and shared/tray suites at earlier head 4c7bb57c. The hosted UI-test failure motivated the follow-up above. Exact-head hardware run 42121609752 passed both RTX jobs: each full build, 4,930 shared tests passed (24 skipped), and 5,062 tray tests passed (0 skipped). Azure built successfully and passed 4,928 shared tests (25 skipped), but one unrelated config-request fixture watchdog timed out; tray tests were not reached. All three hosts passed the transport-classification regression. Attempt 2 cleared the watchdog (4,929 shared passed, 25 skipped), then failed one tray test at CurrentUser DPAPI encryption (5,061 passed, 1 failed). The Azure runner used passwordless S4U login. A credential-backed Password task at the same Limited run level passed a synthetic DPAPI round-trip and the exact failing tray test (1 passed, 0 failed, 0 skipped). The idle runner was switched to that login configuration without rebooting or changing the SSH relay. However, Azure-only attempt 3 failed the same DPAPI test: build passed, shared 4,929 passed / 0 failed / 25 skipped, tray 5,061 passed / 1 failed / 0 skipped. The focused pass did not establish full runner-context recovery: live inspection found the old S4U listener still running after Stop-ScheduledTask. The replacement Password task had exited with a runner-session conflict, so attempt 3 still used the old login. After verifying the runner was idle and stopping only that orphaned listener, the replacement task remained Running and its actual CI worker/testhost shared the new logon session. Azure attempt 4 passed the full build, 4,929 shared tests (25 skipped) and 5,062 tray tests (0 skipped), including the exact formerly failing encryption test. Artifact upload and isolated-settings cleanup passed. No source/test changes, reboot, elevation, or relay changes were needed for this repair. The regular PR run 42120132261 attempt 2 completed with a different unchanged UI-test failure: OnboardingAiPageTests.CurrentModel_NativeUiaRespectsActionAvailability(disabled: True) expected COMException but received ElementNotAvailableException at line 159 (641 passed, 1 failed, 7 skipped). Attempt 1 test-host crash is retained. No further UI retry or test suppression performed; that earlier run’s CI Gate remained failed. The deterministic regression failed both new cases on the previous source, then the patched handshake-observation class passed 12/12, including real loopback connection refusal. Independent workflow and source/test review completed with no outstanding findings.

Earlier hardware run 42116050771 passed full builds on all three hosts and all executed tests on both RTX hosts. The next run reproduced the classifier defect and exposed separate Azure process-fixture timeouts while post-install native-image compilation and Defender saturated its two CPUs. Original failures are retained; no tests or deadlines were weakened.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Required proof pools

  • windows-winui-interactive: native UI Automation invocation and disabled-action rejection on an isolated desktop; not a claim of complete accessibility or visual qualification.
  • windows-11-arm64: native build and test execution on both RTX devices. This does not qualify all ARM64 UI/package behavior.

Validation

  • Integrated-head RTX1 native ARM64 ./build.ps1: passed. Shared dotnet test tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj -r win-arm64 -p:AppendRuntimeIdentifierToOutputPath=false --no-restore: 4,869 passed / 0 failed / 24 skipped. Same command for OpenClaw.Tray.Tests: 5,237 passed / 0 failed / 0 skipped.
  • Integrated-head dotnet test tests/OpenClaw.Tray.UITests -r win-arm64 --filter "FullyQualifiedName~OnboardingAiPageTests&Category!=NativeOnboardingProof": 106 passed / 0 failed / 0 skipped, including enabled invocation and both disabled/unavailable native UIA cases. TRX results, full build transcript, exit receipt, and private-desktop receipts retained. The separate NativeOnboardingProof category is outside this selected suite, not a newly added exclusion.
  • Integrated-head hosted CI: passed, including CI Gate. Downloaded TRX confirms Tray UI 642 passed / 0 failed / 7 not executed, including the original disabled-action case and enabled control; functional UI 19 passed; mock Gateway UI 78 passed, profile app 38 passed, released-profile compatibility 1 passed; accessibility 21 passed / 0 failed / 11 not executed. No retry of the failed old source.
  • Before main integration, an independent local check reproduced the incorrect disabled HRESULT (2 native UIA cases passed, 1 failed); the standard-HRESULT correction passed all 3. This duplicate local patch was preserved as evidence and discarded in favor of the already-merged fix: disabled onboarding actions report the wrong automation error #2050 fix. These are not integrated-head proof claims.
  • Pre-integration full ./build.ps1 passed; shared 4,930 passed / 24 skipped, tray 5,062 passed / 0 skipped, zero failures. This is old head plus the preserved duplicate patch, not integrated-head evidence.
  • An attempted broader original UI suite on the private desktop was cancelled without a result; not counted as a pass.
  • Main-integration AutoReview P0–P3: scoped-clean. actionlint .github/workflows/hardware-validation.yml and git -c core.whitespace=cr-at-eol diff origin/main --check: passed.

Earlier-head evidence below is retained, not relabeled as updated-head CI:

  • Azure login repair proof: synthetic CurrentUser DPAPI protect/unprotect passed under a credential-backed Limited scheduled task. dotnet test tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj -r win-x64 -p:AppendRuntimeIdentifierToOutputPath=false --no-build --no-restore --filter FullyQualifiedName~SettingsPersistenceNotificationTests.TypedStoreRollbackAndExplicitReloadPreserveExternalFieldsAndProtectedSecrets: 1 passed, 0 failed, 0 skipped, exact head 4c7bb57c.

  • PASS: actionlint .github/workflows/hardware-validation.yml

  • PASS: git diff --check

  • PASS: pwsh -NoProfile -File scripts/test-ci-workflow-contract.ps1

  • RED control: detached db205eac plus the new regression, dotnet test tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj -r win-arm64 -p:AppendRuntimeIdentifierToOutputPath=false --filter FullyQualifiedName~TransportFailure_TypedCauseTakesPrecedenceOverEndpointDigits: 2 failed, 0 passed for the intended incorrect Server classification.

  • GREEN focused proof: matching source/test bytes from 4c7bb57c, same native test command with --filter FullyQualifiedName~GatewayHandshakeObservationTests: 12 passed, 0 failed, 0 skipped.

  • Earlier full proof at 9333aa23: both RTX hosts passed ./build.ps1, shared tests 4,928 passed / 24 skipped, and tray tests 5,062 passed / 0 skipped. Azure full build passed; its shared-test failure is retained above. These are not current-head full-run claims.

  • Earlier-head (4c7bb57c) full ./build.ps1: passed on all three hosts. That head's shared/tray suites passed on both RTX hosts with the counts above. Azure attempt 1 failed the request-timeout fixture; attempt 2 passed shared tests but failed DPAPI in one tray test. Credential-backed Limited login passed the focused test, but full Azure attempt 3 still failed the same DPAPI test (shared 4,929 passed / 25 skipped; tray 5,061 passed / 1 failed). Azure attempt 4 subsequently passed: 4,929 shared tests / 25 skipped and 5,062 tray tests / 0 skipped after replacing the surviving S4U listener. Original attempts and TRX artifacts are retained.

Real Behavior Proof

Current-head native proof passed on an isolated RTX1 ARM64 desktop (SSH static host, Interactive/Limited task, no shared input-desktop switch). All 106 selected onboarding cases passed, including the seeded stale-IErrorInfo disabled-action regression and positive invocation control. Hosted CI and CI Gate passed on the same head. Earlier hardware validation at 4c7bb57c is green on all three hosts after the Azure orphaned-listener repair. Main’s #2050 independently reproduced the stale-error failure mechanism; native integration proof now passes; hosted UI, functional, accessibility, and WSL E2E jobs passed.

  • Environment tested: Windows 11 ARM64 RTX devices and Azure Windows 11 x64; focused regression on native RTX ARM64.
  • PR head or commit tested: integrated candidate 3740ef01b65498bbc0303d1e54d00caae3e8c38a (native build/shared/tray/onboarding passed; hosted CI passed); earlier three-host hardware proof 4c7bb57cbe973734d22c9f1856a735613f190d85.
  • Exact steps or command run: commands and counts in Validation; workflow asserts OS/process architecture and source SHA, explicitly builds test projects, and admits only nonempty passing TRX results.
  • Evidence after fix: both socket/TLS endpoint-digit regressions pass, along with the existing handshake-observation class.
  • Observed result: Integrated native onboarding 106/106 passed, including all three native UIA cases. Pre-integration UI contract 2-pass/1-fail baseline became 3/3 pass with the same standard-HRESULT correction. Earlier transport class 12/12 passed; all three hardware hosts passed at 4c7bb57 after the Azure login-process repair.
  • Screenshot or artifact links verified? Yes for the linked run/job pages and downloaded earlier TRX artifacts; no UI screenshot claim.
  • Not verified or blocked: No remaining CI blocker on the updated head. Automated accessibility passed with the counts above; no full manual visual qualification claimed. The original CI worker’s stale-error producer is unknown; the mechanism is independently reproduced in fix: disabled onboarding actions report the wrong automation error #2050. No additional manual live Gateway pairing, MXC, GPU, packaging-release, reboot-recovery, or machine-isolation qualification is claimed beyond the recorded CI jobs.

Security Impact

Execution is limited to explicit dispatch or a maintainer-pushed opt-in branch in the canonical repository. No pull-request trigger, signing secret, or write token. Checkout credentials are not persisted. Tests use run-specific tray settings; this is not machine-level isolation.

  • New permissions or capabilities? Yes: explicit workflows can execute repository code on these dedicated runners; token permissions stay read-only.
  • Secrets or tokens handling changed? No new secrets; checkout credentials are not persisted.
  • New or changed network calls? No product changes; CI performs normal checkout, dependency restore, and artifact upload.
  • Command or tool execution surface changed? Yes: opt-in hardware CI only, not pull-request events.
  • Data access scope changed? No product changes; these remain persistent trusted development hosts.
  • If any answer is Yes, explain the risk and mitigation: only explicit dispatch or the canonical opt-in branch runs this workflow; isolated test settings and scoped cleanup do not claim machine-level isolation.

Compatibility and Migration

Additive opt-in workflow; existing CI is unchanged. Hardware runners require the custom labels declared in actionlint configuration. Azure must remain available for a three-host run.

  • Backward compatible? Yes.
  • Config or environment changes? Yes: dedicated runner labels, native toolchains, Git Bash on PATH, exact-checkout Git trust, non-elevated symlink support, and a working per-user encryption context. Credential-backed login is verified by the full Azure suite after replacing the surviving S4U listener; task configuration alone is insufficient to prove the running process changed.
  • Migration needed? No product migration.
  • If yes, list the exact upgrade steps: operators must prepare the runner prerequisites above and keep selected hosts available before dispatch.

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

@clawsweeper

clawsweeper Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@RomneyDa
RomneyDa marked this pull request as ready for review November 5, 2026 21:48
@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Nov 5, 2026
@clawsweeper

clawsweeper Bot commented Nov 5, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge.

What this changes

Adds an opt-in workflow that builds and tests on two dedicated ARM64 runners and one x64 runner.

Example: A maintainer pushes to ci/hardware-validation.

  • Before: No dedicated workflow validates all three registered hardware runners.
  • After: Each runner checks its native architecture, builds the repository, and requires nonempty passing shared and tray test results.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The patch is focused and source review found no discrete defect, but inaccessible failure details limit confidence in the complete three-host validation.
Proof confidence 🦐 gold shrimp (3/6) This maintainer-authored infrastructure PR is outside the ordinary contributor proof gate; exact-head workflow metadata confirms three real hardware executions and one fully successful job, while this review environment cannot inspect redirected logs.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Maintenance · Worth it: Yes · Fix scope: Complete
User problem: Maintainers lack a dedicated repository-validation lane for the three registered persistent hardware runners.
Reason: The maintainer-authored direction supplies useful native hardware coverage without automatically executing arbitrary pull requests on persistent machines.

Merge readiness

✅ Ready for maintainer review

Keep open: this maintainer-authored PR adds useful, distinct hardware validation absent from main; no definite patch defect was found, but two hardware test failures remain unattributed.

Priority: P2
Reviewed head: db205eac4adfd59e73b5d480da274b6e729208c6

Before merge

None.

Findings

None.

Tests

  • Missing end-to-end proof: The reviewed workflow ran, but blocked log access prevents inspecting test counts and attributing the two failed hardware test steps.
Agent review details

How this fits together

GitHub Actions routes an explicit dispatch or dedicated-branch push to persistent Windows runners, producing build results and uploaded TRX reports.

flowchart TD
A[Manual dispatch or dedicated branch push] --> B[Canonical repository gate]
B --> C[Three Windows hardware runners]
C --> D[Native architecture and prerequisite checks]
D --> E[Repository build]
E --> F[Shared and tray tests]
F --> G[TRX admission and artifacts]
Loading

Technical review

Best possible solution:

Retain the focused opt-in workflow and have the CI owner classify the two hardware test failures before landing.

Do we have a high-confidence way to reproduce the issue?

This is infrastructure work rather than a reported application bug; the exact-head hardware run confirms that all three runner jobs execute.

Is this the best way to solve the issue?

A separate opt-in matrix is a proportionate solution that reuses existing build and test owners while preserving ordinary PR CI.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 4aee38a1bfaa.

Testing

Proof path: shipped entry point.

Security

None.

Evidence

What I checked:

  • Exact introduced change: The pinned introduction adds only the workflow and actionlint runner labels; existing CI and release workflows are unchanged. (.github/workflows/hardware-validation.yml:1, db205eac4adf)
  • Current-head hardware execution: https://github.com/openclaw/openclaw-windows-node/actions/runs/42117800465 executed the reviewed head: all three prerequisite and repository-build steps passed; rtx-2 passed tests, while rtx-1 and azure-mxc-1 failed their test steps. All three uploaded result artifacts. (.github/workflows/hardware-validation.yml:76, db205eac4adf)
  • Failure attribution unavailable: Both failing checks expose only exit code 1 in their annotations. Log downloads redirect to productionresultssa19.blob.core.windows.net, which this environment blocks; neither diff causation nor an unrelated cause is established. (db205eac4adf)
  • Existing validation boundary: Main already builds ARM64 release payloads on windows-11-arm and declares maintainer-scheduled native ARM64 proof, but does not contain this dedicated three-host workflow. (.github/workflows/ci.yml:1210, 4aee38a1bfaa)
  • Discussion and related infrastructure: The discussion contains only review-status bot comments. feat: add clean Windows validation infrastructure #1116 describes a different clean-VM lifecycle and was closed unmerged, so it does not supersede this workflow.
  • Policy and review limits: Read the complete root policy and simplification skill; no scoped .github AGENTS.md or maintainer-notes directory exists. Applied simplification steps 1–3 without finding unnecessary machinery. No helper, second reviewer, build, or test was run. Older-blob reads failed locally, while commit metadata remained available; the checkout remained clean. (AGENTS.md:1, db205eac4adf)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Labels

Label changes:

  • add P2: Dedicated native hardware validation provides concrete maintainer value with a limited, opt-in operational scope.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

  • P2: Dedicated native hardware validation provides concrete maintainer value with a limited, opt-in operational scope.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Inspect the rtx-1 and azure-mxc-1 test diagnostics to distinguish workflow defects from existing test or host failures.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed November 5, 2026, 5:11 PM ET / 22:11 UTC.

@RomneyDa
RomneyDa merged commit 618a5c3 into openclaw:main Nov 7, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant