diff --git a/.claude/skills/playwright-triage/SKILL.md b/.claude/skills/playwright-triage/SKILL.md new file mode 100644 index 00000000..510d87df --- /dev/null +++ b/.claude/skills/playwright-triage/SKILL.md @@ -0,0 +1,102 @@ +--- +name: playwright-triage +description: Triage and fix one microsoft/playwright-cli issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, in this repo or in microsoft/playwright where the source of the CLI is, or an investigation note for it to post on the issue. Only for that workflow. +--- + +# Playwright CLI Issue Triage + +You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of +`microsoft/playwright-cli`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of +[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md), +and then [CLAUDE.md](../../../CLAUDE.md), both in full. The guide says how to triage an issue in any +Playwright repo. CLAUDE.md says how to build, test and lint this one. This file covers what neither +has: where the code is, the checkouts, how to run a published version, and the outcome that this +repo adds, a fix in `microsoft/playwright`. + +## Where the code is + +This repo is the npm package `@playwright/cli`: `playwright-cli.js` with its update check, +`skillCheck.js`, the README, the skill in `skills/playwright-cli/` and some tests of the packaged +CLI. `package.json` pins the alpha of `playwright` and `playwright-core` that the package runs. The +CLI is in `microsoft/playwright`, in `packages/playwright-core/src/tools/`: `cli-client/` has the +commands and their options, `cli-daemon/` has the daemon that keeps the browser open between +commands, and `backend/` has the tools that the commands run. Most bugs and most feature requests +are fixed there. + +Fix a bug here only when the cause is in a file that this repo owns. `skills/playwright-cli/` is a +copy of `packages/playwright-core/src/tools/skills/playwright-cli/`, which `scripts/update.js` +refreshes on a roll, so its fix is upstream too. + +## The checkouts + +`$REPO_DIR` is the checkout of `microsoft/playwright-cli` that the guide describes. `npm ci` ran in +it, and the browsers of the pinned version are installed. The CLI uses Google Chrome from the +runner image by default. When it is missing, install it with `npx playwright install chrome`. The +clone is shallow: run `git fetch --unshallow --tags origin` in it +before you need its history. + +`$PLAYWRIGHT_DIR` is a clone of `microsoft/playwright` at the tip of `main`, next to `$REPO_DIR`. +`npm ci` and `npm run build` ran in it, and Chromium, Firefox and WebKit are installed. It is +tip-of-tree for this repo: reproduce there, as a test in its `tests/mcp/cli-*.spec.ts`, with the +`cli` fixture of `tests/mcp/cli-fixtures.ts`. `npm run test-playwright-cli` runs those tests. Read +its `AGENTS.md` in full before you write the test. It says how to build, test and lint it. The +clone has the full history, without file contents until git needs them, so `git log` and +`git tag --contains` work there. + +The pinned version is older than tip-of-tree. `npm view playwright-core@ gitHead` prints the +upstream commit that an alpha was built from. + +## Fix in microsoft/playwright + +When the fix is in `$PLAYWRIGHT_DIR`, follow the Fix section of the guide there, in place of +`$REPO_DIR`: create the branch `fix-` from `main` in `$PLAYWRIGHT_DIR`, put the regression test +in its `tests/mcp/cli-*.spec.ts`, build and lint as its `AGENTS.md` says, and commit in its +convention, for example `fix(cli): ...`. Then leave the proposal in `$OUTPUT_DIR/upstream-fix/` in +place of `$OUTPUT_DIR/fix/`, with the same files: + +- `fix.patch`: the output of `git diff --binary main HEAD` in `$PLAYWRIGHT_DIR`. +- `pr-body.md`: as the guide says. The `Fixes` line has the full URL of the issue, + `Fixes https://github.com/microsoft/playwright-cli/issues/`, so that the PR closes the issue in + this repo. +- `proposal.json`: as the guide says. Write it last. + +The publish job opens the draft PR in `microsoft/playwright`. The revision workflow of that repo +addresses the review. Leave one proposal: `fix/` or `upstream-fix/`, not both. Do not change +`package.json` here to pick up the fix. It reaches `@playwright/cli` with the next roll. + +**Check the linked PRs in both repos.** A PR in `microsoft/playwright` links this issue with its +full URL or as `microsoft/playwright-cli#`. Search there too: +`repo:microsoft/playwright is:pr "playwright-cli/issues/"` and +`repo:microsoft/playwright is:pr "playwright-cli#"`. A PR there from the team or from +`microsoft-playwright-automation` is a team PR. Verify a PR there in `$PLAYWRIGHT_DIR`. + +**Already fixed** means that tip-of-tree passes and the pinned version fails. The fix reaches users +with the next roll. Find the upstream PR with +`git -C $PLAYWRIGHT_DIR log ..main -- packages/playwright-core/src/tools`, +and name it in the note. + +## Published versions + +To run a published version, for example the version of the reporter or to bisect, create +`$SCRATCH_DIR/playwright-cli-//` and run there: + +```bash +npm init -y +npm i @playwright/cli@ +npx playwright-cli --help +``` + +The fixtures of the repo are not available there. Run the condensed repro as `npx playwright-cli` +commands, or as a script that spawns them, in that directory. Serve the page from a `data:` URL or +from `http.createServer()` in the script. Run `npx playwright-cli close-all` when you are done, so +that no daemon stays open. + +`npm view @playwright/cli@ dependencies.playwright-core` prints the alpha of a release. +Match a regression to upstream commits with +`git -C $PLAYWRIGHT_DIR log .. -- packages/playwright-core/src/tools`. + +## PRs from third parties + +Do not run a PR in this repo that changes `package.json`, `package-lock.json`, `scripts/`, +`.github/`, `.azure-pipelines/` or `.claude/`. For a PR in `microsoft/playwright`, the list in +`$PLAYWRIGHT_DIR/.claude/skills/playwright-triage/SKILL.md` applies. diff --git a/.github/actions/setup-triage-runner/action.yml b/.github/actions/setup-triage-runner/action.yml new file mode 100644 index 00000000..0a9d4025 --- /dev/null +++ b/.github/actions/setup-triage-runner/action.yml @@ -0,0 +1,34 @@ +name: 'Set up the triage runner' +description: > + Prepares the environment that the triage and revision agents work in: the checkout with its + dependencies and the browsers of the pinned version, the Copilot CLI, and next to the checkout a + clone of microsoft/playwright at the tip of main, built, where the source of the CLI is. Exports + PLAYWRIGHT_DIR, the path of that clone, and CI_ENV_EXEC, the wrapper from microsoft/playwright + that runs a command under xvfb on Linux and at normal I/O priority on Windows. +runs: + using: composite + steps: + - uses: microsoft/playwright/.github/actions/ci-env@main + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Install the dependencies and the browsers + shell: bash + run: | + npm ci + npx playwright install --with-deps + # Next to the checkout, where the skills expect it. + - name: Clone and build microsoft/playwright + shell: bash + run: | + dir="$(dirname "${GITHUB_WORKSPACE//\\//}")/playwright" + rm -rf "$dir" + git clone --filter=blob:none https://github.com/microsoft/playwright.git "$dir" + cd "$dir" + npm ci + npm run build + npx playwright install --with-deps chromium firefox webkit + echo "PLAYWRIGHT_DIR=$dir" >> "$GITHUB_ENV" + - name: Install the Copilot CLI + shell: bash + run: npm install -g @github/copilot diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 00000000..bfe08086 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,215 @@ +# Prepares a release. Copilot CLI writes the release notes, as .claude/skills/dev/release.md says, +# without write credentials. Fresh jobs bump the version, open the "chore: mark v" pull +# request with the notes as its description using the app, so CI runs on it, and comment on the +# issue. A maintainer copies the notes into the release draft. +name: Prepare release + +on: + # Apply needs-release to an issue; re-apply it to write the notes again. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue to report on; leave empty for none" + type: string + required: false + default: "" + version: + description: "Version to release; leave empty for the next patch version" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Find the version + if: >- + github.repository == 'microsoft/playwright-cli' && + (github.event_name == 'workflow_dispatch' || github.event.label.name == 'needs-release') + runs-on: ubuntu-latest + permissions: + contents: read + issues: write + outputs: + version: ${{ steps.version.outputs.version }} + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + VERSION: ${{ inputs.version }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Find the next version + id: version + run: | + current=$(node -p "require('./package.json').version") + version=${VERSION:-$(node -p 'const [major, minor, patch] = process.argv[1].split("."); `${major}.${minor}.${+patch + 1}`' "$current")} + [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || { echo "invalid version: $version" >&2; exit 1; } + echo "Releasing v$version; package.json has $current" >> "$GITHUB_STEP_SUMMARY" + echo "version=$version" >> "$GITHUB_OUTPUT" + - name: Post the run link + id: post + if: env.ISSUE != '' + env: + NEXT: ${{ steps.version.outputs.version }} + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to write the release notes of v%s: [agent run](%s)\n" "$NEXT" "$RUN_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + notes: + name: Write the release notes of v${{ needs.start.outputs.version }} + needs: start + runs-on: ubuntu-latest + timeout-minutes: 90 + concurrency: + group: release + cancel-in-progress: false + defaults: + run: + shell: bash + permissions: + contents: read + copilot-requests: write + env: + VERSION: ${{ needs.start.outputs.version }} + outputs: + has_notes: ${{ steps.agent.outputs.has-result }} + steps: + # The full history, for the previous release. No credentials in the checkout. + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + - name: Record the directories + run: | + echo "OUTPUT_DIR=$RUNNER_TEMP/release" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=$RUNNER_TEMP/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=$GITHUB_WORKSPACE" >> "$GITHUB_ENV" + echo "PLAYWRIGHT_DIR=$(dirname "$GITHUB_WORKSPACE")/playwright" >> "$GITHUB_ENV" + mkdir -p "$RUNNER_TEMP/release" "$RUNNER_TEMP/scratch" + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Clone microsoft/playwright and install the Copilot CLI + run: | + git clone --filter=blob:none https://github.com/microsoft/playwright.git "$PLAYWRIGHT_DIR" + npm install -g @github/copilot + - name: Write the notes with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Write the release notes of @playwright/cli v${{ env.VERSION }}. Follow steps 2 to 7 of .claude/skills/dev/release.md. Read the skill in full before you start. + + The workflow does the other steps. A later job does step 1, the version bump, and step 8: it commits, pushes and opens the pull request with your notes as its description. The notes cover the Playwright version that package.json on main pins. Do not change, commit or push files in the checkout, and do not run gh. + + `~/code/playwright` in the skill is $PLAYWRIGHT_DIR here: a clone of microsoft/playwright at the tip of main, with the full history. The checkout of this repo has the full history too. `gh` is not authenticated: read pull requests and issues with the GitHub MCP tools. + + Nobody watches the run: do not ask questions. + + Keep $OUTPUT_DIR/summary.md up to date as you work: the commit window, what you kept, what you dropped and why. Draft the notes in $SCRATCH_DIR. When they are done, write them to $OUTPUT_DIR/release-notes.md, in place of RELEASE_NOTES_v.md, last, in one go. + output-dir: ${{ env.OUTPUT_DIR }} + result-file: release-notes.md + artifact-name: release + github-tools: true + + publish: + name: Open the pull request + needs: [start, notes] + # release-notes.md is written last, so finished notes are published even when the job failed later. + if: ${{ !cancelled() && needs.notes.outputs.has_notes == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.pr.outputs.url }} + env: + VERSION: ${{ needs.start.outputs.version }} + TITLE: "chore: mark v${{ needs.start.outputs.version }}" + BRANCH: release/v${{ needs.start.outputs.version }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + # Pushes as the app, so CI runs on the pull request. + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + token: ${{ steps.app-token.outputs.token }} + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: release + path: ${{ runner.temp }}/release + # Step 1 of the skill. Only the notes come from the agent, as the description. + - name: Bump the version and push the branch + run: | + npm version "$VERSION" --no-git-tag-version --ignore-scripts --allow-same-version + git config user.name "microsoft-playwright-automation[bot]" + git config user.email "203992400+microsoft-playwright-automation[bot]@users.noreply.github.com" + git checkout -b "$BRANCH" + git commit -am "$TITLE" + git push --force origin "$BRANCH" + # The description is the notes verbatim, for a maintainer to copy into the release draft. + - name: Open the pull request + id: pr + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + BASE: ${{ github.ref_name }} + with: + github-token: ${{ steps.app-token.outputs.token }} + script: | + const fs = require('fs'); + const { BASE, BRANCH, TITLE, RUNNER_TEMP } = process.env; + const body = fs.readFileSync(`${RUNNER_TEMP}/release/release-notes.md`, 'utf8').trim(); + if (!body) + throw new Error('release-notes.md is empty'); + const { owner, repo } = context.repo; + const { data: open } = await github.rest.pulls.list({ owner, repo, head: `${owner}:${BRANCH}`, state: 'open' }); + let pr = open[0]; + if (pr) + await github.rest.pulls.update({ owner, repo, pull_number: pr.number, body }); + else + ({ data: pr } = await github.rest.pulls.create({ owner, repo, head: BRANCH, base: BASE, title: TITLE, body })); + core.setOutput('url', pr.html_url); + await core.summary.addRaw(`Pull request: ${pr.html_url}\n`).write(); + + comment: + name: Comment on the issue + needs: [start, notes, publish] + if: ${{ !cancelled() && needs.start.result == 'success' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - name: Post the result + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + ISSUE: ${{ github.event.issue.number || inputs.issue }} + VERSION: ${{ needs.start.outputs.version }} + PR_URL: ${{ needs.publish.outputs.pr }} + START_COMMENT: ${{ needs.start.outputs.comment }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + if [[ -n "$PR_URL" ]]; then + text="I opened the release PR of v$VERSION. Its description has the release notes, ready to copy into the release draft: $PR_URL" + else + text="I couldn't prepare the release of v$VERSION. The agent run has the details." + fi + printf "Hi, I'm the Playwright bot. %s\n\nPrepared by the Playwright bot - [agent run](%s)\n" "$text" "$RUN_URL" > "$RUNNER_TEMP/comment.md" + gh issue comment "$ISSUE" --body-file "$RUNNER_TEMP/comment.md" + if [[ -n "$START_COMMENT" ]]; then + gh api graphql -f query='mutation($id: ID!) { minimizeComment(input: { subjectId: $id, classifier: OUTDATED }) { clientMutationId } }' -f id="$START_COMMENT" \ + || echo "::warning::Could not hide the comment with the run link" + fi diff --git a/.github/workflows/revision.yml b/.github/workflows/revision.yml new file mode 100644 index 00000000..baaf27cb --- /dev/null +++ b/.github/workflows/revision.yml @@ -0,0 +1,137 @@ +# Copilot addresses the maintainers' review comments on a draft PR that the triage workflow opened +# in this repo, without write credentials, on the PR rebased on main. A fresh job replaces the PR's +# branch with the rebased commits and the new one using the app, so CI runs on it. The PRs that the +# triage workflow opens in microsoft/playwright are revised there. The steps that are the same for +# every Playwright repo are actions in microsoft/playwright, and the shared guide they carry says how +# to revise; the prompt below has what is specific to this repo. +name: PR revision + +on: + # Apply needs-revision, needs-revision-mac or needs-revision-win; re-apply it to trigger another pass. + # pull_request_target always runs this file from main; the jobs only accept the triage app's own PRs. + pull_request_target: + types: [labeled] + +permissions: {} + +jobs: + prepare: + name: Collect the maintainers' comments + if: >- + github.repository == 'microsoft/playwright-cli' && + contains(fromJSON('["needs-revision","needs-revision-mac","needs-revision-win"]'), github.event.label.name) && + github.event.pull_request.state == 'open' && + github.event.pull_request.user.login == 'microsoft-playwright-automation[bot]' && + github.event.pull_request.head.repo.full_name == github.repository + runs-on: ubuntu-latest + permissions: + pull-requests: write + env: + PLATFORM: ${{ fromJSON('{"needs-revision":"linux","needs-revision-mac":"mac","needs-revision-win":"windows"}')[github.event.label.name] }} + outputs: + head: ${{ steps.collect.outputs.head }} + base: ${{ steps.collect.outputs.base }} + branch: ${{ steps.collect.outputs.branch }} + platform: ${{ env.PLATFORM }} + steps: + - name: Collect the PR and the maintainers' comments + id: collect + uses: microsoft/playwright/.github/actions/collect-review@main + with: + pull-request: ${{ github.event.pull_request.number }} + - name: Post the run link + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows"}')[env.PLATFORM] }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + body=$(printf "Hi, I'm the Playwright bot. I started to address the review comments on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$RUN_URL") + gh pr comment "$PR" --body "$body" -R "$GITHUB_REPOSITORY" || echo "::warning::Could not post the run link" + + revise: + name: Revise PR ${{ github.event.pull_request.number }} (${{ needs.prepare.outputs.platform }}) + needs: prepare + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[needs.prepare.outputs.platform] }} + timeout-minutes: 240 + concurrency: + group: revision-${{ github.event.pull_request.number }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ needs.prepare.outputs.head }} + BASE_SHA: ${{ needs.prepare.outputs.base }} + TRIAGE_PLATFORM: ${{ needs.prepare.outputs.platform }} + permissions: + copilot-requests: write + outputs: + has_revision: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + persist-credentials: false + - name: Rebase on main + uses: microsoft/playwright/.github/actions/rebase-pr@main + with: + onto: ${{ needs.prepare.outputs.base }} + - name: Record the directories + run: | + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/revision" >> "$GITHUB_ENV" + echo "REVIEW_DIR=${RUNNER_TEMP//\\//}/review" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/revision" "${RUNNER_TEMP//\\//}/scratch" + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: review + path: ${{ runner.temp }}/review + - name: Set up the runner + uses: ./.github/actions/setup-triage-runner + - name: Revise with Copilot CLI + id: agent + # No GitHub tools and no URL access: the agent reads the PR only from the review the prepare job filtered. + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Follow the shared guide revision.md and this repo's CLAUDE.md. Read both in full before starting. $PLAYWRIGHT_DIR is a clone of microsoft/playwright at the tip of main, where the source of the CLI is, built; read it, but do not edit, commit or switch branches in it. + + Pull request: https://github.com/microsoft/playwright-cli/pull/${{ github.event.pull_request.number }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: commit/commit.json + artifact-name: revision + exec: ${{ env.CI_ENV_EXEC }} + + push: + name: Push the revision + needs: [prepare, revise] + # commit.json is written last, so a finished revision is pushed even when the job failed later. + if: ${{ !cancelled() && needs.revise.outputs.has_revision == 'true' }} + runs-on: ubuntu-latest + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: revision + path: ${{ runner.temp }}/revision + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate and push + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/revision/commit + patch: revision.patch + metadata: commit.json + branch: ${{ needs.prepare.outputs.branch }} + onto: ${{ needs.prepare.outputs.base }} diff --git a/.github/workflows/roll.yml b/.github/workflows/roll.yml new file mode 100644 index 00000000..47c846f5 --- /dev/null +++ b/.github/workflows/roll.yml @@ -0,0 +1,302 @@ +# Rolls Playwright to the latest alpha, playwright@next, and opens a pull request. The roll job bumps +# the pins and refreshes the skill, then Copilot CLI updates the README, runs the tests and fixes what +# the roll breaks, as .claude/skills/dev/roll.md says, without write credentials. Fresh jobs open the +# pull request using the app, so CI runs on it, and comment on the issue. +name: Roll Playwright + +on: + # Apply needs-roll to an issue; re-apply it to roll again. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue to report on; leave empty for none" + type: string + required: false + default: "" + version: + description: "Playwright version to roll to; leave empty for playwright@next" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Find the version + if: >- + github.repository == 'microsoft/playwright-cli' && + (github.event_name == 'workflow_dispatch' || github.event.label.name == 'needs-roll') + runs-on: ubuntu-latest + permissions: + contents: read + issues: write + outputs: + old: ${{ steps.versions.outputs.old }} + new: ${{ steps.versions.outputs.new }} + old_commit: ${{ steps.versions.outputs.old_commit }} + new_commit: ${{ steps.versions.outputs.new_commit }} + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + VERSION: ${{ inputs.version }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Read the pinned and the new version + id: versions + run: | + old=$(node -p "require('./package.json').dependencies['playwright-core']") + new=${VERSION:-$(npm view playwright@next version)} + [[ "$new" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || { echo "invalid version: $new" >&2; exit 1; } + new_commit=$(npm view "playwright-core@$new" gitHead) + [[ "$new_commit" =~ ^[0-9a-f]{40}$ ]] || { echo "playwright-core@$new has no gitHead" >&2; exit 1; } + old_commit=$(npm view "playwright-core@$old" gitHead || true) + echo "Pinned $old ($old_commit), rolling to $new ($new_commit)" >> "$GITHUB_STEP_SUMMARY" + { + echo "old=$old" + echo "new=$new" + echo "old_commit=$old_commit" + echo "new_commit=$new_commit" + } >> "$GITHUB_OUTPUT" + - name: Post the run link + id: post + if: env.ISSUE != '' + env: + NEW: ${{ steps.versions.outputs.new }} + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to roll Playwright to %s: [agent run](%s)\n" "$NEW" "$RUN_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + roll: + name: Roll to ${{ needs.start.outputs.new }} + needs: start + if: needs.start.outputs.new != needs.start.outputs.old + runs-on: ubuntu-latest + timeout-minutes: 120 + concurrency: + group: roll + cancel-in-progress: false + defaults: + run: + shell: bash + permissions: + contents: read + copilot-requests: write + env: + OLD: ${{ needs.start.outputs.old }} + NEW: ${{ needs.start.outputs.new }} + OLD_COMMIT: ${{ needs.start.outputs.old_commit }} + NEW_COMMIT: ${{ needs.start.outputs.new_commit }} + outputs: + has_changes: ${{ steps.save.outputs.has_changes }} + complete: ${{ steps.agent.outputs.outcome == 'success' && steps.check.outcome == 'success' && steps.save.outputs.passed == 'true' }} + steps: + # No credentials in the checkout: the agent must not push. + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Record the directories + run: | + echo "OUTPUT_DIR=$RUNNER_TEMP/roll" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=$RUNNER_TEMP/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=$GITHUB_WORKSPACE" >> "$GITHUB_ENV" + echo "PLAYWRIGHT_DIR=$(dirname "$GITHUB_WORKSPACE")/playwright" >> "$GITHUB_ENV" + mkdir -p "$RUNNER_TEMP/roll" "$RUNNER_TEMP/scratch" + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Clone microsoft/playwright at the new version + run: | + git clone --filter=blob:none --no-checkout https://github.com/microsoft/playwright.git "$PLAYWRIGHT_DIR" + git -C "$PLAYWRIGHT_DIR" checkout -q "$NEW_COMMIT" + # Steps 1 to 3 of .claude/skills/dev/roll.md. + - name: Bump the pins and refresh the skill + run: | + npm pkg set "dependencies.playwright=$NEW" "dependencies.playwright-core=$NEW" "devDependencies.@playwright/test=$NEW" + npm install + node scripts/update.js + - name: Install the Copilot CLI + run: npm install -g @github/copilot + - name: Roll with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Roll Playwright in @playwright/cli from ${{ env.OLD }} to ${{ env.NEW }}. Follow .claude/skills/dev/roll.md, and read CLAUDE.md for how to test. + + The workflow already did steps 1 to 3 of the skill: package.json pins ${{ env.NEW }}, `npm install` updated package-lock.json, and `node scripts/update.js` refreshed skills/playwright-cli/. $PLAYWRIGHT_DIR is a clone of microsoft/playwright at ${{ env.NEW_COMMIT }}, the commit that the new version was built from; the old version was built from ${{ env.OLD_COMMIT }}. Start with the README update: `git diff -- skills/` shows what changed in the skill. + + When a test fails, find out why before you change anything. `git -C $PLAYWRIGHT_DIR log ${{ env.OLD_COMMIT }}..${{ env.NEW_COMMIT }} -- packages/playwright-core/src/tools tests/mcp` lists the upstream changes. When an upstream change is intended, update the test in this repo to the new behavior. When it is a regression, do not hide it: leave the test as it is, and describe the regression and the upstream commit that caused it. Change only README.md and files in skills/ and tests/: the workflow rejects changes to other files, except the pins in package.json and package-lock.json. + + The last step is for the workflow: do not create a branch, commit, push or run gh. A later job commits your changes and opens the pull request. Nobody watches the run: do not ask questions. + + Keep $OUTPUT_DIR/summary.md up to date as you work. When you finish, write the pull request description to $OUTPUT_DIR/pr-body.md: what changed in the skill and the README, the test results, and for each test that you changed, why, with the upstream commit. Then write $OUTPUT_DIR/roll.json last: {"passed": true} when `npm test` passes, {"passed": false} otherwise. + output-dir: ${{ env.OUTPUT_DIR }} + result-file: roll.json + artifact-name: roll + # The skill says to check that playwright-core rolled: the CLI runs it, and the tests use @playwright/test. + - name: Check the roll + id: check + continue-on-error: true + run: | + node -e ' + const pkg = require("./package.json"); + const pins = [pkg.dependencies.playwright, pkg.dependencies["playwright-core"], pkg.devDependencies["@playwright/test"], require("playwright-core/package.json").version]; + if (pins.some(pin => pin !== process.argv[1])) + throw new Error(`package.json pins and node_modules have ${pins.join(", ")}`); + ' "$NEW" + node playwright-cli.js --help > /dev/null + - name: Save the changes + id: save + run: | + git add -A + # Against the checked-out commit, in case the agent committed anyway. + git diff --cached --binary "$GITHUB_SHA" > "$OUTPUT_DIR/roll.patch" + if [[ -s "$OUTPUT_DIR/roll.patch" ]]; then + echo "has_changes=true" >> "$GITHUB_OUTPUT" + else + echo "has_changes=false" >> "$GITHUB_OUTPUT" + fi + passed=false + if [[ -f "$OUTPUT_DIR/roll.json" ]]; then + passed=$(node -p 'JSON.parse(require("fs").readFileSync(process.argv[1], "utf8")).passed === true' "$OUTPUT_DIR/roll.json" || echo false) + fi + echo "passed=$passed" >> "$GITHUB_OUTPUT" + { + echo '```' + git diff --cached --stat "$GITHUB_SHA" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + - name: Upload the roll + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: roll + path: ${{ env.OUTPUT_DIR }} + overwrite: true + if-no-files-found: warn + + publish: + name: Open the pull request + needs: [start, roll] + if: ${{ !cancelled() && needs.roll.outputs.has_changes == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.pr.outputs.url }} + env: + TITLE: "chore: roll Playwright to ${{ needs.start.outputs.new }}" + BRANCH: roll/${{ needs.start.outputs.new }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + # Pushes as the app, so CI runs on the pull request. + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: roll + path: ${{ runner.temp }}/roll + # Never execute code or scripts from the roll in the app job. + - name: Push the branch + run: | + git apply --index "$RUNNER_TEMP/roll/roll.patch" + # A roll changes only what a roll changes. --no-renames lists a renamed file under its old + # name too. + git diff --cached --name-only --no-renames > "$RUNNER_TEMP/changed.txt" + if grep -v -E '^(package\.json|package-lock\.json|README\.md)$|^(skills|tests)/' "$RUNNER_TEMP/changed.txt"; then + echo "The roll changes files outside package.json, package-lock.json, README.md, skills/ and tests/." >&2 + exit 1 + fi + if grep -E '(^|/)(AGENTS|CLAUDE)\.md$' "$RUNNER_TEMP/changed.txt"; then + echo "The roll changes agent instructions." >&2 + exit 1 + fi + if git diff --cached --summary | grep ' mode 120000 '; then + echo "The roll adds a symbolic link." >&2 + exit 1 + fi + git config user.name "microsoft-playwright-automation[bot]" + git config user.email "203992400+microsoft-playwright-automation[bot]@users.noreply.github.com" + git checkout -b "$BRANCH" + git commit -m "$TITLE" + git push --force origin "$BRANCH" + - name: Open the pull request + id: pr + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + BASE: ${{ github.ref_name }} + COMPLETE: ${{ needs.roll.outputs.complete }} + OLD_COMMIT: ${{ needs.start.outputs.old_commit }} + NEW_COMMIT: ${{ needs.start.outputs.new_commit }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + with: + github-token: ${{ steps.app-token.outputs.token }} + script: | + const fs = require('fs'); + const { BASE, BRANCH, TITLE, COMPLETE, OLD_COMMIT, NEW_COMMIT, RUN_URL, RUNNER_TEMP } = process.env; + const file = `${RUNNER_TEMP}/roll/pr-body.md`; + const parts = [fs.existsSync(file) ? fs.readFileSync(file, 'utf8').trim() : 'The agent wrote no description.']; + if (COMPLETE !== 'true') + parts.push(`**The roll is not complete:** the tests, the agent or the checks failed. See the [roll run](${RUN_URL}).`); + if (OLD_COMMIT) + parts.push(`Upstream changes: https://github.com/microsoft/playwright/compare/${OLD_COMMIT}...${NEW_COMMIT}`); + parts.push(`Rolled by the [roll workflow](${RUN_URL}).`); + const body = parts.join('\n\n'); + const { owner, repo } = context.repo; + const { data: open } = await github.rest.pulls.list({ owner, repo, head: `${owner}:${BRANCH}`, state: 'open' }); + let pr = open[0]; + if (pr) + await github.rest.pulls.update({ owner, repo, pull_number: pr.number, body }); + else + ({ data: pr } = await github.rest.pulls.create({ owner, repo, head: BRANCH, base: BASE, title: TITLE, body, draft: COMPLETE !== 'true' })); + core.setOutput('url', pr.html_url); + await core.summary.addRaw(`Pull request: ${pr.html_url}\n`).write(); + + comment: + name: Comment on the issue + needs: [start, roll, publish] + if: ${{ !cancelled() && needs.start.result == 'success' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - name: Post the result + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + ISSUE: ${{ github.event.issue.number || inputs.issue }} + NEW: ${{ needs.start.outputs.new }} + ROLL_RESULT: ${{ needs.roll.result }} + COMPLETE: ${{ needs.roll.outputs.complete }} + PR_URL: ${{ needs.publish.outputs.pr }} + START_COMMENT: ${{ needs.start.outputs.comment }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + if [[ "$ROLL_RESULT" == "skipped" ]]; then + text="main already pins Playwright $NEW. There is nothing to roll." + elif [[ -n "$PR_URL" && "$COMPLETE" == "true" ]]; then + text="I rolled Playwright to $NEW, and the tests pass: $PR_URL" + elif [[ -n "$PR_URL" ]]; then + text="I rolled Playwright to $NEW, but the roll is not complete: the tests, the agent or the checks failed. The pull request says what is left: $PR_URL" + else + text="I couldn't roll Playwright to $NEW. The agent run has the details." + fi + printf "Hi, I'm the Playwright bot. %s\n\nRolled by the Playwright bot - [agent run](%s)\n" "$text" "$RUN_URL" > "$RUNNER_TEMP/comment.md" + gh issue comment "$ISSUE" --body-file "$RUNNER_TEMP/comment.md" + if [[ -n "$START_COMMENT" ]]; then + gh api graphql -f query='mutation($id: ID!) { minimizeComment(input: { subjectId: $id, classifier: OUTDATED }) { clientMutationId } }' -f id="$START_COMMENT" \ + || echo "::warning::Could not hide the comment with the run link" + fi diff --git a/.github/workflows/triage.yml b/.github/workflows/triage.yml new file mode 100644 index 00000000..22b29c77 --- /dev/null +++ b/.github/workflows/triage.yml @@ -0,0 +1,276 @@ +# Copilot triages and fixes an issue, or works on a task given by hand, without write credentials. +# Fresh jobs post the run link on the issue, open the completed fix as a draft PR using the app, so +# CI runs on it, and comment on the issue. The source of the CLI is in microsoft/playwright, +# so most fixes open there, and that repo's revision workflow addresses the review. The steps that +# are the same for every Playwright repo are actions in microsoft/playwright. The skill in +# .claude/skills/playwright-triage/ has what is specific to this repo. +name: Issue triage + +on: + # Apply needs-triage, needs-triage-mac or needs-triage-win; re-apply it to trigger another pass. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue number to triage; leave empty to work on the task alone" + type: string + required: false + default: "" + platform: + description: "Platform to triage on" + type: choice + required: true + default: linux + options: + - linux + - mac + - windows + task: + description: "Task to direct the triage, such as what to focus on; the whole job when there is no issue" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Post the run link on the issue + # The triage job's condition, for runs that have an issue. + if: >- + github.repository == 'microsoft/playwright-cli' && + ((github.event_name == 'workflow_dispatch' && inputs.issue != '') || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + runs-on: ubuntu-latest + permissions: + issues: write + outputs: + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - name: Post the run link + id: post + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to triage this issue on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$WORKFLOW_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + triage: + name: Triage ${{ (github.event.issue.number || inputs.issue) && format('issue {0}', github.event.issue.number || inputs.issue) || 'task' }} (${{ inputs.platform || github.event.label.name }}) + if: >- + github.repository == 'microsoft/playwright-cli' && + (github.event_name == 'workflow_dispatch' || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + # Falls back to a runner name for unrelated labels, where the job is skipped anyway. + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name]] || 'ubuntu-latest' }} + timeout-minutes: 240 + concurrency: + group: triage-${{ github.event.issue.number || inputs.issue || github.run_id }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + TRIAGE_PLATFORM: ${{ inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name] }} + TASK: ${{ inputs.task }} + permissions: + copilot-requests: write + outputs: + base: ${{ steps.base.outputs.sha }} + upstream_base: ${{ steps.upstream-base.outputs.sha }} + setup: ${{ steps.setup.outcome }} + agent: ${{ steps.agent.outputs.outcome }} + has_fix: ${{ steps.agent.outputs.has-result }} + has_upstream_fix: ${{ steps.upstream-fix.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + persist-credentials: false + - name: Record the base and the directories + id: base + run: | + if [[ -n "$ISSUE" ]]; then + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + elif [[ -z "${TASK//[[:space:]]/}" ]]; then + echo "give an issue, a task or both" >&2; exit 1 + fi + echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/triage" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/triage" "${RUNNER_TEMP//\\//}/scratch" + - name: Set up the runner + id: setup + uses: ./.github/actions/setup-triage-runner + - name: Record the base of microsoft/playwright + id: upstream-base + run: echo "sha=$(git -C "$PLAYWRIGHT_DIR" rev-parse HEAD)" >> "$GITHUB_OUTPUT" + - name: Write the prompt + id: prompt + run: | + delimiter="PROMPT_$RANDOM$RANDOM" + { + echo "prompt<<$delimiter" + echo "Follow .claude/skills/playwright-triage/SKILL.md. Read it, the guide and the CLAUDE.md" + echo "it points to in full before starting." + if [[ -n "$ISSUE" ]]; then + printf '\nIssue: https://github.com/%s/issues/%s\n' "$GITHUB_REPOSITORY" "$ISSUE" + fi + if [[ -n "$TASK" ]]; then + printf '\nTask: %s\n' "$TASK" + fi + echo "$delimiter" + } >> "$GITHUB_OUTPUT" + - name: Triage with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: ${{ steps.prompt.outputs.prompt }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: fix/proposal.json + artifact-name: triage + github-tools: true + exec: ${{ env.CI_ENV_EXEC }} + - name: Check the fix in microsoft/playwright + id: upstream-fix + run: | + if [[ -f "$OUTPUT_DIR/upstream-fix/proposal.json" ]]; then + echo "has-result=true" >> "$GITHUB_OUTPUT" + echo "Fix proposed in microsoft/playwright" >> "$GITHUB_STEP_SUMMARY" + else + echo "has-result=false" >> "$GITHUB_OUTPUT" + fi + + publish: + name: Open draft pull request + needs: triage + # proposal.json is written last, so a finished proposal is published even when the job failed later. + if: ${{ !cancelled() && needs.triage.outputs.has_fix == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.publish.outputs.pull-request }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.triage.outputs.base }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: triage + path: ${{ runner.temp }}/triage + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate, push and open draft + id: publish + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/triage/fix + patch: fix.patch + metadata: proposal.json + body-file: pr-body.md + branch: triage/${{ github.event.issue.number || inputs.issue || 'task' }}/${{ github.run_id }}-${{ github.run_attempt }} + commit-body: "${{ (github.event.issue.number || inputs.issue) && format('Fixes: https://github.com/{0}/issues/{1}', github.repository, github.event.issue.number || inputs.issue) || '' }}" + pull-request: true + + publish-upstream: + name: Open draft pull request in microsoft/playwright + needs: triage + # proposal.json is written last, so a finished proposal is published even when the job failed later. + if: ${{ !cancelled() && needs.triage.outputs.has_upstream_fix == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.pr.outputs.url }} + env: + BRANCH: triage/playwright-cli-${{ github.event.issue.number || inputs.issue || 'task' }}/${{ github.run_id }}-${{ github.run_attempt }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + owner: microsoft + repositories: playwright + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: microsoft/playwright + ref: ${{ needs.triage.outputs.upstream_base }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: triage + path: ${{ runner.temp }}/triage + # Never execute code or scripts from the proposed changes in the app job. publish-patch opens + # pull requests in the repo of the workflow only, so it only pushes, and the next step opens it. + - name: Validate and push + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/triage/upstream-fix + patch: fix.patch + metadata: proposal.json + branch: ${{ env.BRANCH }} + commit-body: "${{ (github.event.issue.number || inputs.issue) && format('Fixes: https://github.com/{0}/issues/{1}', github.repository, github.event.issue.number || inputs.issue) || '' }}" + - name: Open draft + id: pr + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + DIR: ${{ runner.temp }}/triage/upstream-fix + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + with: + github-token: ${{ steps.app-token.outputs.token }} + script: | + const fs = require('fs'); + const path = require('path'); + const { BRANCH, DIR, RUN_URL } = process.env; + const { title } = JSON.parse(fs.readFileSync(path.join(DIR, 'proposal.json'), 'utf8')); + const description = fs.readFileSync(path.join(DIR, 'pr-body.md'), 'utf8').trim(); + const { data: pr } = await github.rest.pulls.create({ + owner: 'microsoft', repo: 'playwright', head: BRANCH, base: 'main', title, + body: `${description}\n\nPrepared by the [triage workflow of ${context.repo.owner}/${context.repo.repo}](${RUN_URL}).`, + draft: true, + }); + core.setOutput('url', pr.html_url); + await core.summary.addRaw(`Draft pull request: ${pr.html_url}\n`).write(); + + comment: + name: Comment on the issue + needs: [start, triage, publish, publish-upstream] + # Runs after every triage run that has an issue, including failed ones, and whatever publish did. + if: ${{ !cancelled() && needs.triage.result != 'skipped' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + # A failed triage run may not have uploaded anything. + continue-on-error: true + with: + name: triage + path: ${{ runner.temp }}/triage + - name: Post the triage comment + uses: microsoft/playwright/.github/actions/triage-comment@main + with: + issue: ${{ github.event.issue.number || inputs.issue }} + dir: ${{ runner.temp }}/triage + footer: Triaged on ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} by the Playwright bot + job-result: ${{ needs.triage.result }} + setup-outcome: ${{ needs.triage.outputs.setup }} + agent-outcome: ${{ needs.triage.outputs.agent }} + setup-failure-reason: installing the dependencies, building microsoft/playwright or downloading the browsers failed + pull-request: ${{ needs.publish.outputs.pr || needs.publish-upstream.outputs.pr }} + publish-result: ${{ (needs.publish.result == 'failure' || needs.publish-upstream.result == 'failure') && 'failure' || '' }} + start-comment: ${{ needs.start.outputs.comment }} diff --git a/CLAUDE.md b/CLAUDE.md index b8a60e07..889fc9fc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,3 +1,22 @@ +## Build and test + +The CLI source lives in `microsoft/playwright`, in `packages/playwright-core/src/tools/`. This repo +ships `@playwright/cli`: `playwright-cli.js` runs the CLI of the `playwright-core` alpha that +`package.json` pins, and adds the update check, with `skillCheck.js`. `skills/playwright-cli/` is a +copy of the skill that `playwright-cli install --skills` writes, and `node scripts/update.js` +refreshes it. + +```bash +npm ci +npm test # tests/integration.spec.ts +npx playwright test -g "open data URL" +``` + +There is no build step and no lint. Tests are in `tests/integration.spec.ts`. They run +`playwright-cli.js` with `runCli()` in the output directory of the test, and the `npmRegistry` +fixture serves a fake registry for the update check. Add a case to the test whose setup is closest +instead of a new test. + ## Commit Convention Semantic commit messages: `label(scope): description` @@ -28,3 +47,19 @@ EOF Never add Co-Authored-By agents in commit message. Branch naming for issue fixes: `fix-` + +## Rolling Playwright and preparing a release + +See [.claude/skills/dev/roll.md](.claude/skills/dev/roll.md) and [.claude/skills/dev/release.md](.claude/skills/dev/release.md). + +## Bots + +Labels start GitHub Actions workflows that run the Copilot CLI and open pull requests as the +`microsoft-playwright-automation` app: + +- `needs-triage`, `needs-triage-mac`, `needs-triage-win` on an issue: [triage.yml](.github/workflows/triage.yml) reproduces and fixes it, with the skill in [.claude/skills/playwright-triage/](.claude/skills/playwright-triage/SKILL.md). A fix in `microsoft/playwright` opens as a draft PR there. +- `needs-revision`, `needs-revision-mac`, `needs-revision-win` on a PR that triage opened here: [revision.yml](.github/workflows/revision.yml) addresses the maintainers' review comments. +- `needs-roll` on an issue: [roll.yml](.github/workflows/roll.yml) rolls Playwright to `playwright@next`, as the roll skill says. +- `needs-release` on an issue: [release.yml](.github/workflows/release.yml) opens the `chore: mark v` PR with the release notes as its description, as the release skill says. Copy the notes into the release draft. + +Apply a label again to start another run. [bots.md](https://github.com/microsoft/playwright/blob/main/.github/workflows/bots.md) in `microsoft/playwright` describes triage and revision.