Drive verify-graph with the Playwright CLI

software/application-software/argos/workspace-defaults/workspace.yml

@@ -4,6 +4,7 @@ plugins:                  # `- <scope>[.<set>]`: plugin sets to enable; CI lists
 park:                     # `- <folder>`: root folders that park with this workspace on a switch (.delphi/park.sh), besides the ones it always parks
   - pictures
   - .playwright-mcp
+  - .playwright-cli
 # .delphi/setup.sh checks out everything below (git-ignored); re-run it after any change.
 repos:                    # `<name>: <git-url>`: code you change; bare store at repos/<name>, default branch at worktrees/<name>/<branch>
   argos: https://github.com/Northeastern-Electric-Racing/Argos.git

software/application-software/argos/workspace-inherit/.claude/skills/run-local/SKILL.md

@@ -9,6 +9,6 @@ Run the branches the user is asking about: a PR or ticket means its branch (make
 bash scripts/run-local.sh <branch>...
 ```
 
-It starts the backend if it's down (`client-dev`, or `scylla-dev` plus `cargo run` when a target changes `scylla-server/`), and serves each target's client on the next free port. With no branches it reports what's running.
+It starts the backend if it's down (`client-dev`, or `scylla-dev` plus `cargo run` when a target changes `scylla-server/`), and serves each target's client on the next free port; parallel runs (one per session or agent) each get their own. With no branches it reports what's running.
 
 Report each branch's client URL and the backend. A failed compile or a stale-backend warning is a finding: show it to the user. Stop a client with `kill <pid>` from its line.

software/application-software/argos/workspace-inherit/.claude/skills/run-local/scripts/serve.sh

@@ -18,9 +18,18 @@ if [ -f "$key.pid" ] && kill -0 "$(cat "$key.pid")" 2>/dev/null && up "$(cat "$k
   exit 0
 fi
 
+# Claim the port under a lock, or parallel calls all pick the same one while ng serve starts. A port is
+# taken once a client listens on it or a live serve.sh has claimed it. mkdir is the lock (no flock on
+# macOS); it is held only for the claim, so nothing started below can keep it.
+lock=$state/port.lock
+n=0; until mkdir "$lock" 2>/dev/null; do n=$((n + 1)); [ $n -lt 100 ] || rm -rf "$lock"; sleep 0.1; done # stale after 10s
+trap 'rmdir "$lock" 2>/dev/null || true' EXIT
+claimed() { for f in "$state"/*.port; do [ -f "$f" ] && [ "$(cat "$f")" = "$1" ] && kill -0 "$(cat "${f%.port}.pid")" 2>/dev/null && return; done; return 1; }
 port=
-for p in $(seq 4200 4210); do up "$p" || { port=$p && break; }; done
+for p in $(seq 4200 4210); do up "$p" || claimed "$p" || { port=$p && break; }; done
 [ -n "$port" ] || { echo "serve: no free port in 4200-4210" >&2 && exit 1; }
+echo $$ >"$key.pid" && echo "$port" >"$key.port" # claimed by this script until ng serve's pid replaces it
+rmdir "$lock" && trap - EXIT
 
 cd "$wt/angular-client"
 [ -x node_modules/.bin/ng ] || npm ci --no-audit --no-fund >"$log" 2>&1 ||

software/application-software/argos/workspace-inherit/.claude/skills/verify-graph/SKILL.md

@@ -1,21 +1,22 @@
 ---
 name: verify-graph
-description: Playwright MCP verification of the graph page — modes, controls, topic selection, and data rendering. Use after any change to graph-page components, graph rendering, data flow, or mode switching logic.
+description: Playwright CLI verification of the graph page — modes, controls, topic selection, and data rendering. Use after any change to graph-page components, graph rendering, data flow, or mode switching logic.
 user-invocable: true
 ---
 
-Get the ticket's branch running with `run-local <branch>`; that branch's client URL is `BASE_URL`. Historical runs with data must exist. Screenshots go to `pictures/<branch>/verify-graph/NN-<name>.png` (gitignored).
+Get the branch's client running with `run-local <branch>`; its URL is `BASE_URL`. Then run the base checks:
 
-Per check: `browser_navigate` → `browser_snapshot` (assert) → `browser_click` or `browser_run_code` → `browser_take_screenshot`. Two clicks need `browser_run_code` with a body from `helpers.js`: **selectRun** ("Select this Run" sits under a PrimeNG overlay) and **selectTopic** (a tree click on a leaf navigates away). Run the groups in order; stop and fix at the first failure.
+```
+bash scripts/check.sh <branch> "$BASE_URL"
+```
 
-| # | Do | Assert |
-|---|---|---|
-| 1 | Open `{BASE_URL}/graph` | Header "Real Time"; Pause, Time Range, Select Run, Clear Graph; "X (sec):", Y Min/Max, "Select Range"; no Realtime button |
-| 2 | Expand BMS › Pack, selectTopic "Voltage", wait 3s | URL has `?topics=BMS/Pack/Voltage`; a series is drawn |
-| 3 | Pause↔Play; Time Range↔Point Range; Tog Sidebar twice | Labels flip ("X (points):" in point range); sidebar hides, then returns |
-| 4 | Select Run, then selectRun, wait 3s | Header "Run #N"; Realtime shown; no Pause or Time Range; Y Min/Max remain; x-axis spans more than 30s |
-| 5 | Realtime; open Select Range, pick "5 minutes" | 7 options (1, 2, 5, 10, 15, 30 min, 1 hour); header "Hist Range"; Realtime shown, no Pause; x-axis about 5 min |
-| 6 | Cycle Realtime → Run → Realtime → "1 minute" → Realtime | Headers match each mode; no console errors (`browser_console_messages`) |
-| 7 | In real time, wait 10s | x-axis about 30s, not the whole session (historical auto-fit doesn't leak) |
+It seeds a 50-hour run spanning two midnights (once, shared by parallel checks), opens a `playwright-cli` session named after the branch (`scripts/open.sh`: output to `pictures/<branch>/`, config to `.playwright-cli/<branch>/`), and runs `scripts/check.js`. That covers: page loads in real time, a topic from the tree draws live, the seeded run, historical Quick Select, back to real time, and no console errors beyond develop's NG0100. It prints `[x]`/`[ ]` per check and exits 1 on a failure. Each `look:` line names a screenshot and what it should show: open it and judge it, since only a picture shows whether a chart is right.
 
-Report a checklist (one line per group, `[x]` or `[ ]` with the failure) and the screenshot folder.
+Go further where the diff points, by hand in the same session (`P="playwright-cli -s=<branch>"`): controls the diff touches (Pause↔Play, Time Range↔Point Range, Tog Sidebar, Tog Mult Yaxis, Y Min/Max, Custom range, Clear Graph), zoom, pan and hover, several topics, or mode switching for console errors. Reproduce a failed check the same way before fixing it; `check.js` has its steps.
+
+- Act: `$P click "button:has-text('Pause')"`, `role=combobox[name='Quick Select']`, `role=option[name='5 minutes']`, or a ref from `$P snapshot`.
+- Assert: `$P find "<text>"`, `$P snapshot`, `$P console error`; longer Playwright code with `$P run-code "async (page) => {…}"`, or `--filename` for a file kept in `.playwright-cli/<branch>/`.
+- The seeded run: click Select Run, then `bash scripts/select-run.sh <branch> $(bash scripts/seed.sh)`.
+- Screenshots: `$P screenshot --filename=pictures/<branch>/verify-graph/NN-<name>.png` (gitignored).
+
+When a hand check keeps coming back, add it to `check.js`. Report `check.sh`'s lines plus one line per hand check, and the screenshot folder. `$P close` at the end.

software/application-software/argos/workspace-inherit/.claude/skills/verify-graph/scripts/check.js

@@ -0,0 +1,82 @@
+// The base verify-graph checks, for check.sh to run with `playwright-cli run-code` (it fills in __URL__,
+// __RUN__ and __OUT__). Each check asserts what the DOM can show; what only a picture can show (a series
+// drawn, the x-axis span) is a screenshot with a `look:` line for the agent to judge.
+async (page) => {
+  const URL = '__URL__', RUN = __RUN__, OUT = '__OUT__';
+  const lines = [], errors = [];
+  page.on('console', (m) => {
+    if (m.type() === 'error' && !m.text().includes('NG0100')) errors.push(m.text().split('\n')[0].slice(0, 160));
+  }); // develop already logs NG0100 (ExpressionChangedAfterItHasBeenChecked)
+  const text = (t) => page.getByText(t, { exact: true });
+  const button = (name) => page.getByRole('button', { name, exact: true });
+  const tree = page.getByRole('tree');
+  const absent = async (name) => { if (await button(name).isVisible()) throw new Error(`"${name}" is shown`); };
+  const shot = async (name, look) => {
+    await page.screenshot({ path: `${OUT}/${name}.png` });
+    lines.push(`    look: ${OUT}/${name}.png: ${look}`);
+  };
+  const check = async (name, steps) => {
+    const at = lines.length;
+    try {
+      await steps();
+      lines.splice(at, 0, `[x] ${name}`);
+    } catch (e) {
+      lines.push(`[ ] ${name}: ${String(e.message).split('\n')[0]}`);
+    }
+  };
+
+  await check('1 page loads in real time', async () => {
+    await page.goto(`${URL}/graph`);
+    await text('Real Time').waitFor({ timeout: 30000 });
+    for (const b of ['Pause', 'Time Range', 'Select Run', 'Clear Graph', 'Custom']) await button(b).waitFor({ timeout: 2000 });
+    await page.getByRole('combobox', { name: 'Quick Select' }).waitFor({ timeout: 2000 });
+    await absent('Realtime');
+  });
+  await check('2 topic from the tree draws live', async () => {
+    const node = (name) => tree.getByRole('treeitem').filter({ has: page.getByText(name, { exact: true }) }).last(); // innermost
+    for (const name of ['BMS', 'Segment_Total_Volt']) await node(name).locator('.p-tree-node-toggle-button').first().click();
+    await tree.getByText('0', { exact: true }).first().click();
+    await page.waitForURL(/topics=BMS%2FSegment_Total_Volt%2F0/, { timeout: 5000 });
+    await page.waitForTimeout(3000);
+    await shot('02-live-topic', 'a live series is drawn');
+  });
+  await check(`3 seeded run #${RUN}`, async () => {
+    await button('Select Run').click();
+    await page.locator('.run-card').first().waitFor({ timeout: 10000 });
+    await page.evaluate((id) => {
+      const card = [...document.querySelectorAll('.run-card')]
+        .find((c) => [...c.querySelectorAll('*')].some((e) => !e.childElementCount && e.textContent.trim() === 'Run #' + id));
+      const pick = card?.parentElement?.querySelector('.run-select-button button');
+      if (!pick) throw new Error(`no Run #${id} in the Select Run dialog`);
+      pick.dispatchEvent(new MouseEvent('click', { bubbles: true, cancelable: true })); // under a PrimeNG overlay
+    }, RUN);
+    await page.locator('.run-card').first().waitFor({ state: 'detached', timeout: 5000 });
+    await text(`Run #${RUN}`).waitFor({ timeout: 10000 });
+    await button('Realtime').waitFor({ timeout: 2000 });
+    await absent('Pause');
+    await page.waitForTimeout(3000);
+    await shot('03-seeded-run', 'x-axis spans 50 hours, with a dated day-boundary line at each of the two midnights');
+  });
+  await check('4 historical Quick Select', async () => {
+    await button('Realtime').click();
+    await text('Real Time').waitFor({ timeout: 5000 });
+    await page.getByRole('combobox', { name: 'Quick Select' }).click();
+    const options = await page.getByRole('option').count();
+    if (options !== 11) throw new Error(`${options} Quick Select options, not 11`);
+    await page.getByRole('option', { name: '5 minutes', exact: true }).click();
+    await text('Historical').waitFor({ timeout: 5000 });
+    await absent('Pause');
+    await page.waitForTimeout(3000);
+    await shot('04-historical-5-min', 'x-axis about 5 minutes, or since the data starts');
+  });
+  await check('5 back to real time', async () => {
+    await button('Realtime').click();
+    await text('Real Time').waitFor({ timeout: 5000 });
+    await page.waitForTimeout(10000);
+    await shot('05-real-time-after', 'x-axis about 30 seconds, not the whole session');
+  });
+  await check('6 no new console errors', async () => {
+    if (errors.length) throw new Error(`${errors.length}, first: ${errors[0]}`);
+  });
+  return lines.join('\n');
+}

software/application-software/argos/workspace-inherit/.claude/skills/verify-graph/scripts/check.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+# check.sh <branch> <base-url>: run the base verify-graph checks (check.js) in <branch>'s playwright-cli
+# session, opening it if needed, against the seeded multi-day run. Prints one `[x]`/`[ ]` line per check
+# and `look:` lines for screenshots in pictures/<branch>/verify-graph/; exits 1 if a check failed.
+set -euo pipefail
+here=$(cd "$(dirname "$0")" && pwd)
+branch=${1:?usage: check.sh <branch> <base-url>}; url=${2:?usage: check.sh <branch> <base-url>}; url=${url%/}
+run=$(bash "$here/seed.sh")
+out=${DELPHI_ROOT:?unset: start the session at the workspace root}/pictures/$branch/verify-graph
+playwright-cli list | grep -q "^- $branch:" || bash "$here/open.sh" "$branch" "$url/graph" >/dev/null
+code=$(sed -e "s|__URL__|$url|; s|__RUN__|$run|; s|__OUT__|$out|" "$here/check.js")
+result=$(playwright-cli -s="$branch" --raw run-code "$code" | node -e 'process.stdout.write(JSON.parse(require("fs").readFileSync(0, "utf8")))')
+echo "$result"
+! grep -q '^\[ \]' <<<"$result"

software/application-software/argos/workspace-inherit/.claude/skills/verify-graph/scripts/open.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+# open.sh <branch> <url>: open a playwright-cli session named <branch> on <url>, so parallel checks don't
+# share a browser. Its output (snapshots, console logs, unnamed screenshots) goes to pictures/<branch>/;
+# its config, and any throwaway Playwright code, to .playwright-cli/<branch>/. Both are gitignored. Uses
+# the preinstalled Chromium in cloud containers (/opt/pw-browsers); elsewhere, Playwright's own
+# (`playwright-cli install-browser`).
+set -euo pipefail
+branch=${1:?usage: open.sh <branch> <url>}; url=${2:?usage: open.sh <branch> <url>}
+root=${DELPHI_ROOT:?unset: start the session at the workspace root}
+out=$root/pictures/$branch scratch=$root/.playwright-cli/$branch
+mkdir -p "$out" "$scratch"
+launch=; [ -x /opt/pw-browsers/chromium ] && launch=',"launchOptions":{"executablePath":"/opt/pw-browsers/chromium","headless":true}'
+printf '{"browser":{"browserName":"chromium"%s},"outputDir":"%s"}\n' "$launch" "$out" >"$scratch/cli.config.json"
+playwright-cli -s="$branch" open --config="$scratch/cli.config.json" "$url"

software/application-software/argos/workspace-inherit/.claude/skills/verify-graph/scripts/seed.sh

@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+# seed.sh: make sure the local database holds verify-graph's multi-day run, and print its id. The run is
+# 50 hours of BMS/Segment_Total_Volt/0, one point every 30s, from noon UTC four days ago, so it crosses
+# two midnights in any time zone and stays out of the 24-hour Quick Select window. Parallel checks share
+# one run: it is made once, under a lock, and found by its notes after that.
+set -euo pipefail
+db=$(docker ps --filter name=odyssey-db --format '{{.Names}}' | head -1)
+[ -n "$db" ] || { echo "seed: no odyssey-db container; start the backend with run-local first" >&2 && exit 1; }
+sql() { docker exec -i "$db" psql -U postgres -v ON_ERROR_STOP=1 -qAt "$@"; }
+tag='verify-graph multi-day seed'
+state=${TMPDIR:-/tmp}; state=${state%/}/argos-run-local; mkdir -p "$state"
+lock=$state/seed.lock
+n=0; until mkdir "$lock" 2>/dev/null; do n=$((n + 1)); [ $n -lt 600 ] || rm -rf "$lock"; sleep 0.1; done # stale after 60s
+trap 'rmdir "$lock" 2>/dev/null || true' EXIT
+id=$(sql -c "select \"runId\" from run where notes = '$tag' limit 1")
+[ -n "$id" ] || id=$(sql <<SQL
+insert into data_type (name, unit) values ('BMS/Segment_Total_Volt/0', 'V') on conflict do nothing;
+with r as (
+  insert into run ("driverName", "locationName", notes, time)
+  values ('verify-graph', 'seed', '$tag', date_trunc('day', now()) - interval '84 hours')
+  returning "runId", time
+), d as (
+  insert into data (values, time, "dataTypeName", "runId")
+  select array[400 + 20 * sin(s / 3600.0)]::real[], (extract(epoch from r.time) * 1e6)::bigint + s * 1000000::bigint,
+    'BMS/Segment_Total_Volt/0', r."runId"
+  from r, generate_series(0, 50 * 3600, 30) s
+)
+select "runId" from r;
+SQL
+)
+echo "$id"

software/application-software/argos/workspace-inherit/.claude/skills/verify-graph/scripts/select-run.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+# select-run.sh <branch> <run-id>: with the Select Run dialog open in <branch>'s session, pick run
+# <run-id>. The carousel autoplays and its "Select this Run" button sits under a PrimeNG overlay, so this
+# finds the run's card by id and clicks its button with a synthetic event instead of `playwright-cli click`.
+set -euo pipefail
+branch=${1:?usage: select-run.sh <branch> <run-id>}; id=${2:?usage: select-run.sh <branch> <run-id>}
+case $id in '' | *[!0-9]*) echo "select-run: run id must be a number, got '$id'" >&2 && exit 1 ;; esac
+playwright-cli -s="$branch" run-code "async (page) => {
+  await page.waitForSelector('.run-card');
+  const found = await page.evaluate((id) => {
+    const card = [...document.querySelectorAll('.run-card')]
+      .find((c) => [...c.querySelectorAll('*')].some((e) => !e.childElementCount && e.textContent.trim() === 'Run #' + id));
+    const button = card?.parentElement?.querySelector('.run-select-button button');
+    button?.dispatchEvent(new MouseEvent('click', { bubbles: true, cancelable: true }));
+    return !!button;
+  }, $id);
+  if (!found) throw new Error('select-run: no Run #$id in the Select Run dialog');
+}"

software/application-software/argos/workspaces/argos-dev/workspace.yml

@@ -6,6 +6,7 @@ plugins:                  # `- <scope>[.<set>]`: plugin sets to enable; CI lists
 park:                     # `- <folder>`: root folders that park with this workspace on a switch (.delphi/park.sh), besides the ones it always parks
   - pictures
   - .playwright-mcp
+  - .playwright-cli
 # .delphi/setup.sh checks out everything below (git-ignored); re-run it after any change.
 repos:                    # `<name>: <git-url>`: code you change; bare store at repos/<name>, default branch at worktrees/<name>/<branch>
   argos: https://github.com/Northeastern-Electric-Racing/Argos.git

software/application-software/argos/workspaces/experimental/workspace.yml

@@ -6,6 +6,7 @@ plugins:                  # `- <scope>[.<set>]`: plugin sets to enable; CI lists
 park:                     # `- <folder>`: root folders that park with this workspace on a switch (.delphi/park.sh), besides the ones it always parks
   - pictures
   - .playwright-mcp
+  - .playwright-cli
 # .delphi/setup.sh checks out everything below (git-ignored); re-run it after any change.
 repos:                    # `<name>: <git-url>`: code you change; bare store at repos/<name>, default branch at worktrees/<name>/<branch>
   argos: https://github.com/Northeastern-Electric-Racing/Argos.git