Skip to content

Desktop testing: declare and select WDIO cases by input lane, platform and scope #4464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jason Morse (JasonVMo) merged 3 commits into from
Nov 7, 2026
Merged

Desktop testing: declare and select WDIO cases by input lane, platform and scope #4464

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f5a90cd
WDIO descriptor and desktopIt helper
JasonVMo Nov 6, 2026
d888442
Merge origin/main into user/jasonvmo/input-selection
JasonVMo Nov 6, 2026
451e1cb
Select and account for desktop WDIO cases by input lane
JasonVMo Nov 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/input-lanes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@fluentui-react-native/desktop-driver': patch
'@fluentui-react-native/desktop-test': patch
'@fluentui-react-native/storybook-desktop': patch
---

Add modality-aware desktop test descriptors: `desktopIt` and `desktopDescribe` declare `requires`, `input`, `platforms`, `scope`, and known gaps for Storybook cases and, through `desktop-test/jasmine`, WebdriverIO Jasmine specs. `--input accessible|physical` (or `DESKTOP_INPUT`) runs one input lane, selects the cases that apply, masks the host for the accessible lane, and reports the lane and what it excluded.
5 changes: 5 additions & 0 deletions .changeset/native-core-desktop-it.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@fluentui-react-native/native-core': patch
---

Declare the Callout and FocusZone desktop tests with `desktopIt` and `requires`, so each runs in the input lane it needs and the Win32 Tab-exit gap is declared instead of skipped at run time.
26 changes: 15 additions & 11 deletions apps/E2E/src/ButtonV1/specs/ButtonV1.spec.win.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
import { desktopDescribe, desktopIt } from '@fluentui-react-native/desktop-test/jasmine';

import { BUTTON_ACCESSIBILITY_LABEL, BUTTON_TEST_COMPONENT_LABEL } from '../../ButtonLegacy/consts';
import { BUTTON_A11Y_ROLE, Keys, Attribute } from '../../common/consts';
import ButtonV1PageObject from '../pages/ButtonV1PageObject';

// Before testing begins, allow up to 60 seconds for app to open
describe('ButtonV1 Testing Initialization', () => {
it('Wait for app load', async () => {
desktopDescribe('ButtonV1 Testing Initialization', () => {
desktopIt('Wait for app load', async () => {
expect(await ButtonV1PageObject.waitForInitialPageToDisplay()).toBeTrue();
});

it('Click and navigate to ButtonV1 test page', async () => {
// Navigation scrolls the page list with the wheel whenever the page button is out of view.
desktopIt('Click and navigate to ButtonV1 test page', { requires: ['click', 'wheel'] }, async () => {
/* Click on component button to navigate to test page */
expect(await ButtonV1PageObject.navigateToPageAndLoadTests()).toBeTrue();

Expand All @@ -21,18 +24,19 @@ describe('ButtonV1 Testing Initialization', () => {
});
});

describe('ButtonV1 Accessibility Testing', () => {
// The setup scrolls the test element into view, which can inject native wheel input, so this group is physical.
desktopDescribe('ButtonV1 Accessibility Testing', { requires: ['wheel'] }, () => {
beforeEach(async () => {
await ButtonV1PageObject.scrollToTestElement();
});

it('Validate "accessibilityRole" value defaults to "ControlType.Button".', async () => {
desktopIt('Validate "accessibilityRole" value defaults to "ControlType.Button".', async () => {
await expect(
await ButtonV1PageObject.compareAttribute(ButtonV1PageObject._primaryComponent, Attribute.AccessibilityRole, BUTTON_A11Y_ROLE),
).toBeTruthy();
});

it('Set "accessibilityLabel" prop. Validate "accessibilityLabel" value propagates to "Name" element attribute.', async () => {
desktopIt('Set "accessibilityLabel" prop. Validate "accessibilityLabel" value propagates to "Name" element attribute.', async () => {
await expect(
await ButtonV1PageObject.compareAttribute(
ButtonV1PageObject._primaryComponent,
Expand All @@ -42,7 +46,7 @@ describe('ButtonV1 Accessibility Testing', () => {
).toBeTruthy();
});

it('Do NOT set "accessibilityLabel" prop. Validate "Name" element attribute defaults to the button label.', async () => {
desktopIt('Do NOT set "accessibilityLabel" prop. Validate "Name" element attribute defaults to the button label.', async () => {
await expect(
await ButtonV1PageObject.compareAttribute(
ButtonV1PageObject._secondaryComponent,
Expand All @@ -53,7 +57,7 @@ describe('ButtonV1 Accessibility Testing', () => {
});
});

describe('ButtonV1 Functional Testing', () => {
desktopDescribe('ButtonV1 Functional Testing', { requires: ['wheel'] }, () => {
/* Scrolls and waits for the Button to be visible on the Test Page */
beforeEach(async () => {
await ButtonV1PageObject.scrollToTestElement();
Expand All @@ -63,7 +67,7 @@ describe('ButtonV1 Functional Testing', () => {
await ButtonV1PageObject.resetTest();
});

it('Click primary button. Validate onClick() callback was fired.', async () => {
desktopIt('Click primary button. Validate onClick() callback was fired.', { requires: ['physical-click'] }, async () => {
await ButtonV1PageObject.click(ButtonV1PageObject._primaryComponent);
await expect(
await ButtonV1PageObject.waitForOnClickCallbackToFire(`The primary button failed to fire an onClick callback with a mouse click.`),
Expand All @@ -74,7 +78,7 @@ describe('ButtonV1 Functional Testing', () => {
.toBeFalsy();
});

it('Type "Enter" on primary button. Validate onClick() callback was fired.', async () => {
desktopIt('Type "Enter" on primary button. Validate onClick() callback was fired.', { requires: ['keyboard'] }, async () => {
await ButtonV1PageObject.sendKeys(ButtonV1PageObject._primaryComponent, [Keys.ENTER]);
await expect(
await ButtonV1PageObject.waitForOnClickCallbackToFire(
Expand All @@ -87,7 +91,7 @@ describe('ButtonV1 Functional Testing', () => {
.toBeFalsy();
});

it('Type "Space" on primary button. Validate onClick() callback was fired.', async () => {
desktopIt('Type "Space" on primary button. Validate onClick() callback was fired.', { requires: ['keyboard'] }, async () => {
await ButtonV1PageObject.sendKeys(ButtonV1PageObject._primaryComponent, [Keys.SPACE]);

await expect(
Expand Down
35 changes: 18 additions & 17 deletions packages/agentic/desktop-driver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,23 +139,24 @@ The [reference index](references/README.md) describes each package-owned guide.

## Public entry points

| Export | Purpose |
| --------------- | ----------------------------------------------------------------------------------------- |
| Package root | Native-helper build/resolution, `DesktopHost`, server, low-level client, and common types |
| `/authoring` | Serializable story plans, selectors, capabilities, and result types |
| `/wdio` | WebdriverIO connection, custom commands, assertions, and story-plan runner |
| `/client` | Low-level typed W3C client |
| `/authority` | Grants, private connection descriptors, and macOS owner-validated storage |
| `/native` | Native-helper build, resolution, and verification without loading the package root |
| `/server` | Embeddable remote end, session manager, and target registry |
| `/agent` | Bounded JSON-safe inspection, action, and evidence API |
| `/operations` | Versioned agent operation catalog, executor, and result types |
| `/artifacts` | Confined, atomic evidence persistence |
| `/runner` | Plan selection, sharding, execution, and result classification |
| `/storybook` | Story manifest and orchestration contracts, without a Storybook dependency |
| `/testing` | Fake host, fake Storybook orchestrator, and protocol harnesses |
| `/cli` | Embeddable Commander command and CLI runner |
| `/cli/reporter` | Shared reporter mode resolution and structured output framing |
| Export | Purpose |
| ------------------ | ----------------------------------------------------------------------------------------- |
| Package root | Native-helper build/resolution, `DesktopHost`, server, low-level client, and common types |
| `/authoring` | Serializable story plans, selectors, capabilities, result types, and case descriptors |
| `/wdio` | WebdriverIO connection, custom commands, assertions, and story-plan runner |
| `/client` | Low-level typed W3C client |
| `/authority` | Grants, private connection descriptors, and macOS owner-validated storage |
| `/native` | Native-helper build, resolution, and verification without loading the package root |
| `/server` | Embeddable remote end, session manager, and target registry |
| `/agent` | Bounded JSON-safe inspection, action, and evidence API |
| `/operations` | Versioned agent operation catalog, executor, and result types |
| `/artifacts` | Confined, atomic evidence persistence |
| `/runner` | Plan selection, sharding, execution, and result classification |
| `/storybook` | Story manifest and orchestration contracts, without a Storybook dependency |
| `/testing` | Fake host, fake Storybook orchestrator, and protocol harnesses |
| `/cli` | Embeddable Commander command and CLI runner |
| `/cli/reporter` | Shared reporter mode resolution and structured output framing |
| `/authoring/input` | Descriptor vocabulary and scope rules with no runtime imports (loads from source) |

The service implements native application semantics rather than pretending to
be a browser. Unsupported browser features return `unsupported operation`
Expand Down
5 changes: 5 additions & 0 deletions packages/agentic/desktop-driver/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@
"import": "./lib/authoring/index.js",
"default": "./src/authoring/index.ts"
},
"./authoring/input": {
"types": "./lib/authoring/input.d.ts",
"import": "./lib/authoring/input.js",
"default": "./src/authoring/input.ts"
},
"./agent": {
"types": "./lib/agent/index.d.ts",
"import": "./lib/agent/index.js",
Expand Down
38 changes: 19 additions & 19 deletions packages/agentic/desktop-driver/references/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,25 +120,25 @@ handoff are still work in progress; see [current status](current-status.md).

## Module ownership

| Module | Responsibility |
| ------------------ | ------------------------------------------------------------------ |
| `src/protocol` | W3C parsing, actions, capabilities, timeouts, and errors |
| `src/server` | HTTP routing, target registry, sessions, windows, and elements |
| `src/host` | Platform-neutral `DesktopHost` and target contracts |
| `src/hosts/fake` | Deterministic in-memory provider |
| `src/hosts/native` | Long-lived helper process and `DesktopHost` adapter |
| `src/native` | Build, cache, resolution, verification, manifests, and wire types |
| `native/macos` | Swift operating-system implementation |
| `native/windows` | Shared C++ Windows and Win32 implementation |
| `src/client` | Low-level typed W3C client |
| `src/wdio` | Sanctioned WebdriverIO integration and custom commands |
| `src/authoring` | Serializable story plans, selectors, capabilities, and results |
| `src/runner` | Filtering, sharding, execution, and result classification |
| `src/artifacts` | Confined atomic evidence |
| `src/agent` | Bounded JSON-safe operations |
| `src/operations` | Agent operation catalog, executor, references, ledger, and journal |
| `src/testing` | Fake harnesses and Storybook test doubles |
| `src/cli` | Commander parsing, JSON output, and process exit adaptation |
| Module | Responsibility |
| ------------------ | -------------------------------------------------------------------------------- |
| `src/protocol` | W3C parsing, actions, capabilities, timeouts, and errors |
| `src/server` | HTTP routing, target registry, sessions, windows, and elements |
| `src/host` | Platform-neutral `DesktopHost` and target contracts |
| `src/hosts/fake` | Deterministic in-memory provider |
| `src/hosts/native` | Long-lived helper process and `DesktopHost` adapter |
| `src/native` | Build, cache, resolution, verification, manifests, and wire types |
| `native/macos` | Swift operating-system implementation |
| `native/windows` | Shared C++ Windows and Win32 implementation |
| `src/client` | Low-level typed W3C client |
| `src/wdio` | Sanctioned WebdriverIO integration and custom commands |
| `src/authoring` | Serializable story plans, selectors, capabilities, results, and case descriptors |
| `src/runner` | Filtering, sharding, execution, and result classification |
| `src/artifacts` | Confined atomic evidence |
| `src/agent` | Bounded JSON-safe operations |
| `src/operations` | Agent operation catalog, executor, references, ledger, and journal |
| `src/testing` | Fake harnesses and Storybook test doubles |
| `src/cli` | Commander parsing, JSON output, and process exit adaptation |

## Deliberate boundaries

Expand Down
35 changes: 34 additions & 1 deletion packages/agentic/desktop-driver/references/contributing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ change-oriented guidance.
| `src/native` | Source build, cache, selection, verification, and native artifact types |
| `native/macos` | Swift implementation and self-test |
| `native/windows` | Shared C++ Windows/Win32 implementation and self-test |
| `src/authoring` | Static JSON plan and result schemas |
| `src/authoring` | Static JSON plan and result schemas, and the case descriptor |
| `src/runner` | Plan selection, capability checks, execution, and classification |
| `src/wdio` | Sanctioned WebdriverIO connection and commands |
| `src/client` | Low-level typed W3C client |
Expand All @@ -38,6 +38,39 @@ Do not duplicate native build, cache, or verification logic in Storybook
consumers. Do not move Storybook manifest generation or app lifecycle into this
package.

## Case descriptors and input lanes

`src/authoring/input.ts` and `src/authoring/selection.ts` are the one
implementation of the test case descriptor (`requires`, `input`, `platforms`,
`scope`, and declared gaps). Storybook extraction and its runner, and the
desktop-test Jasmine adapter, all call them; do not reimplement validation,
merging, or a decision in a consumer. Keep both files pure data and functions,
with no Node, React, or WebdriverIO imports, because stories and workers load
them. `input.ts` also has no runtime imports at all (its one import is a type),
and the `./authoring/input` subpath exports it directly: desktop-test's config
loader reads scope domains from it, and a config file is loaded from source
through a CommonJS fallback that cannot resolve this package's `.js`
specifiers, so keep it that way.

- Add a requirement to `desktopRequirements`, to the physical or accessible
requirement set in `input.ts` when it implies an input, and to the
`requirementFeature` map in `selection.ts` for the host feature that backs it,
together. A requirement whose feature a host did not report is unmet.
- `mergeDesktopCaseOptions` only narrows: requirements union, `platforms` and
`scope` intersect, and a nested declaration can never widen or change its
group's input. An overlap that is empty is an error because the case could
never run. A group cannot carry a gap.
- `decideDesktopCase` judges platform, then scope, then the lane, so a case is
excluded for one reason. It resolves a declared gap too, so an expired
quarantine stops a run before anything executes.
- `resolveDesktopClickMode` lets a lane decide the click mode outright. That is
how one `click` requirement is an accessibility click in the accessible lane
and a physical click in the physical lane.
- `accessibleLaneMaskedFeatures` (`keyboard`, `physicalClick`, `wheel`) is what
an accessible lane disables through the native host's `disabledInputFeatures`,
and `unmaskedAccessibleFeatures` fails closed when a host reports any of them
as still available. Change the set together with the hosts that apply it.

## Behavioral rules

- Reserve a target before asynchronous launch or attach.
Expand Down
27 changes: 18 additions & 9 deletions packages/agentic/desktop-driver/references/test-integration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,15 +98,24 @@ exits, including on assertion failures, timeouts, or crashes. WebdriverIO's
sets `waitforInterval` (100 ms) and `waitforTimeout` (5 s) explicitly; without
an interval, `waitUntil` and `waitFor*` stop after their first unsatisfied poll.

Storybook's preferred top-level `wdio` functions and named function collections
use this pattern with Node's built-in test runner. Each function receives the
target `platform` (`macos`, `windows`, or `win32`), and each named case gets an
independent worker/session. Native feature flags are typed on the WebdriverIO
browser's `furn:features` capability. Button now uses this authoring pattern
instead of serializable plans. Extraction, native-bundle stripping, runner
configuration, and source freshness belong to
`@fluentui-react-native/storybook-desktop`,
not this Storybook-independent package. Static plans below are unchanged.
Storybook's preferred top-level `wdio` export is an array of `desktopIt` cases,
which declare the input and platforms they need. Its older functions and named
function collections use the same pattern with Node's built-in test runner.
Each function receives the target `platform` (`macos`, `windows`, or `win32`),
and each named case gets an independent worker/session. Native feature flags are
typed on the WebdriverIO browser's `furn:features` capability. Button now uses
this authoring pattern instead of serializable plans. Extraction, native-bundle
stripping, runner configuration, and source freshness belong to
`@fluentui-react-native/storybook-desktop`, not this Storybook-independent
package. Static plans below are unchanged.

The descriptor that `desktopIt` accepts (`requires`, `input`, `platforms`,
`scope`, and declared gaps) is defined and decided in `./authoring`, so Storybook
and `desktop-test` share one implementation. A runner that executes one input lane
pins `furn:clickMode` with `resolveDesktopClickMode`: `element.click()` is then an
accessibility click in the accessible lane and a physical click in the physical
lane. `missingDesktopRequirements` judges a case's `requires` against the
session's `furn:features`, and judges a `click` requirement by the pinned mode.

## Author portable story plans

Expand Down
Loading
Loading