Skip to content

Add GitHub Actions CI workflow alongside CircleCI - #9868

Open
camd wants to merge 1 commit into
masterfrom
camd/github-actions-ci
Open

camd wants to merge 1 commit into
masterfrom
camd/github-actions-ci

Conversation

@camd

@camd camd commented Sep 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

First step of migrating CI from CircleCI to GitHub Actions. This PR adds a ci.yml workflow that mirrors every CircleCI job except deploy, and leaves .circleci/config.yml untouched so both systems run side by side.

GitHub Actions job Mirrors CircleCI job What it runs
javascript-tests javascript-tests pnpm test:ci + Codecov
javascript-integration-tests javascript-integration-tests Playwright (Firefox), report uploaded as artifact
javascript-lint javascript-lint pnpm lint, pnpm markdownlint
builds builds tox -e docs, tox -e linters
python-tests-general / -perf / -frontend / -telemetry same tox -e docker-* + Codecov
test-docker-build test-docker-build BuildKit build of docker/Dockerfile, no push

Job names deliberately match the CircleCI names so required status checks can be swapped one-for-one at cutover.

Other changes

  • .github/actions/setup-pnpm: composite action replacing the CircleCI setup-pnpm command (pnpm via corepack, Node from engines, store cache).
  • .github/dependabot.yml: track github-actions versions.

Manual preparation, by phase

Everything below is done outside the repo, by hand, by someone with the right access. Nothing in this section is automated by any PR. Names only are shown here; values are never displayed by CircleCI and must be looked up or regenerated at the source given.

Check what is already set at any time (names only, never values):

gh secret list --repo mozilla/treeherder
gh variable list --repo mozilla/treeherder

As of 2026-09-15 both lists are empty.

Phase 1: this PR

The workflow needs no repo configuration to pass: every job is green with zero secrets or variables set. The single manual step is for coverage.

Kind Name Where the value comes from Who
Secret CODECOV_TOKEN https://app.codecov.io/gh/mozilla/treeherder/settings > "Repository Upload Token" Codecov admin + GitHub repo admin
gh secret set CODECOV_TOKEN --repo mozilla/treeherder   # prompts on stdin

Why it is needed: Codecov has protected-branch mode on for this repo, so tokenless uploads are rejected with Token required because branch is protected. Jobs still pass because fail_ci_if_error: false, and CircleCI keeps uploading in the meantime, so this does not block landing. Set it at or before merge, then re-run the workflow on this PR and confirm the Codecov step logs successful uploads for both the javascript and python flags.

Do not change required status checks on master in this phase. They stay pointed at the ci/circleci: * contexts; the Actions jobs are informational until cutover.

Do not touch .circleci/. Both systems run on every PR until they have agreed for at least a week or ~20 merged PRs.

Validation only possible after merge:

  • Fork PRs cannot see ci.yml until it is on master. On the first fork PR after merge, confirm the workflow runs and all jobs pass with no secrets (the Codecov step falls back to tokenless and is rejected, which is expected and harmless on forks).
  • cancel-in-progress verified 2026-09-22: a force-push to Add GitHub Actions deploy workflow alongside CircleCI #9884 (same workflow file) while run 35754368085 was in flight left that run cancelled and started a fresh one.

Phase 2: deploy workflow (follow-up PR)

The deploy workflow builds the production image and pushes it to Docker Hub and GCP Artifact Registry, exactly as the CircleCI deploy job does today. It reads the following. All must exist before the workflow can even dry-run.

Secrets (GitHub repo admin):

Name Where the value comes from
DOCKER_USER The Docker Hub account CircleCI pushes as (owner of the mozilla/treeherder image; visible on hub.docker.com)
DOCKER_PASS Mint a new Docker Hub access token with Read & Write scope (Account settings > Personal access tokens). Do not try to reuse CircleCI's; it cannot be read back
gh secret set DOCKER_USER --repo mozilla/treeherder
gh secret set DOCKER_PASS --repo mozilla/treeherder

Variables, plain non-secret values (GitHub repo admin, values from a GCP admin):

Name Where the value comes from
DOCKERHUB_REPO Image name CircleCI pushes to, almost certainly mozilla/treeherder
GCP_GAR_REGISTRY Registry host, e.g. us-docker.pkg.dev: gcloud artifacts repositories list --project=<gar-project>, take <location>-docker.pkg.dev
GCP_PROJECT_ID_OF_GCP_GAR Project id holding the treeherder-prod Artifact Registry repository
GCP_WIP_PROJECT_NUMBER Numeric project number of the project holding the Workload Identity Pool: gcloud projects describe <project-id> --format='value(projectNumber)'
GCP_WIP_ID Pool id: gcloud iam workload-identity-pools list --location=global --project=<pool-project>
GCP_WIP_PROVIDER_ID The new GitHub provider created in the GCP step below (suggested id github)
GCP_GAR_PUSHER_SERVICE_ACCOUNT_EMAIL The service account CircleCI impersonates; find it on the CircleCI provider's IAM bindings or gcloud iam service-accounts list
gh variable set <NAME> --repo mozilla/treeherder --body <value>

The workflow assembles the Artifact Registry image path as ${GCP_GAR_REGISTRY}/${GCP_PROJECT_ID_OF_GCP_GAR}/treeherder-prod/treeherder, matching CircleCI, and the OIDC provider path as projects/${GCP_WIP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${GCP_WIP_ID}/providers/${GCP_WIP_PROVIDER_ID}.

GCP: add a GitHub OIDC provider to the existing Workload Identity Pool (needs roles/iam.workloadIdentityPoolAdmin on the pool project and roles/iam.serviceAccountAdmin on the pusher service account). The pool currently trusts only CircleCI's issuer; GitHub signs with a different one, so a second provider is required. The attribute condition pins it to this repository.

POOL_PROJECT=<project id that holds the pool>
POOL_ID=<GCP_WIP_ID>
SA_EMAIL=<GCP_GAR_PUSHER_SERVICE_ACCOUNT_EMAIL>
POOL_PROJECT_NUMBER=$(gcloud projects describe "$POOL_PROJECT" --format='value(projectNumber)')

gcloud iam workload-identity-pools providers create-oidc github \
  --project="$POOL_PROJECT" --location=global \
  --workload-identity-pool="$POOL_ID" \
  --display-name="GitHub Actions" \
  --issuer-uri="https://token.actions.githubusercontent.com" \
  --attribute-mapping="google.subject=assertion.sub,attribute.repository=assertion.repository,attribute.repository_owner=assertion.repository_owner,attribute.ref=assertion.ref" \
  --attribute-condition="assertion.repository == 'mozilla/treeherder'"

gcloud iam service-accounts add-iam-policy-binding "$SA_EMAIL" \
  --project="$(echo "$SA_EMAIL" | sed 's/.*@\(.*\)\.iam\.gserviceaccount\.com/\1/')" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/$POOL_PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/attribute.repository/mozilla/treeherder"

The service account keeps its existing Artifact Registry writer role. Leave the CircleCI provider in place until cutover.

Optional, recommended: create a GitHub environment named deploy and put the Docker Hub secrets there instead of at repo level, so only the deploy workflow can read them. The deploy workflow has a commented-out environment: deploy line for this.

Record the current required status checks on master (repo admin, Settings > Branches or Settings > Rules). Nothing changes yet; cutover needs the list. The branch-protection API returns 404 for non-admin tokens, so this has to be read from the UI.

Verification once all of the above exists: run the deploy workflow via workflow_dispatch with its push input left false. That exercises version.json, the image build, the Docker Hub login, and the GCP token exchange without publishing anything.

Phase 3: cutover (third PR)

Repo changes in that PR: delete .circleci/config.yml, swap the README badge, update ui/contribute.json and a couple of doc comments, and rename the builds job to docs-and-linters.

By hand, in the same sitting as merging that PR (otherwise PRs block on checks that never report):

  1. Repo admin swaps required status checks from ci/circleci: <job> to the bare job names: javascript-tests, javascript-integration-tests, javascript-lint, docs-and-linters (formerly builds), python-tests-general, python-tests-perf, python-tests-frontend, python-tests-telemetry, test-docker-build.
  2. CircleCI: "Stop Building" in project settings. Then remove the CircleCI GitHub App / OAuth integration from the mozilla org only if Treeherder was its last user (check with Mozilla's GitHub admins first).
  3. Docker Hub: revoke the access token CircleCI was using.
  4. GCP admin: delete the CircleCI OIDC provider from the Workload Identity Pool.
  5. CircleCI: delete the project context / environment variables (DOCKER_USER, DOCKER_PASS, DOCKERHUB_REPO, GCP_*, CODECOV_TOKEN).

Rollback before Phase 3 is "do nothing": CircleCI remains the system of record. After Phase 3, rollback is reverting the cutover PR and re-adding the required checks.

Notes for reviewers

  • ubuntu-latest is 4 vCPU for public repos, matching the CircleCI large class the Jest/Playwright worker counts were tuned for.
  • Nothing in this workflow needs a secret to pass, so fork PRs run it normally. CODECOV_TOKEN is the one exception and is covered above; an earlier version of this description said tokenless uploads would work, which turned out to be wrong for this repo.
  • deploy (Docker Hub + GCP Artifact Registry) comes in a follow-up PR once a GitHub OIDC provider is added to the GCP Workload Identity Pool.
  • Cutover (delete .circleci/, README badge, required checks) is a third PR after both systems have agreed for a while.
  • builds is a misnomer: it only runs tox -e docs and tox -e linters, and the rspack production build already happens inside test-docker-build via the Dockerfile. The name is kept here so the required status check swaps one-for-one. It will be renamed to docs-and-linters at cutover, when the required check is updated anyway.
  • The four Python jobs race to save the same pip cache key on a cold cache (Failed to save: ... another job may be creating this cache). Harmless; one wins and later runs hit it.

@netlify

netlify Bot commented Sep 12, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for treeherder ready!

Name Link
🔨 Latest commit 2f76ca7
🔍 Latest deploy log https://app.netlify.com/projects/treeherder/deploys/6ace57bbe070d6621a722346
😎 Deploy Preview https://deploy-preview-9868--treeherder.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@camd

camd commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator Author

First run is green on all nine jobs: https://github.com/mozilla/treeherder/actions/runs/34714644932

Job GitHub Actions
javascript-lint 23s
javascript-tests 58s (1093 passed, 5 skipped)
javascript-integration-tests 1m10s (19 passed, report artifact uploaded)
python-tests-telemetry 3m27s
python-tests-frontend 3m55s
builds 4m07s
test-docker-build 4m09s
python-tests-general 5m01s
python-tests-perf 5m24s

One thing to sort out before merge or shortly after: Codecov rejected the uploads with Token required because branch is protected, so this repo's Codecov project does not allow tokenless uploads. A CODECOV_TOKEN repository secret (from the Codecov project settings page) is needed for coverage to land. The jobs do not fail on it (fail_ci_if_error: false), so coverage is the only thing missing until the secret is added.

@codecov-commenter

codecov-commenter commented Sep 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.18%. Comparing base (3cf6bff) to head (2f76ca7).

Additional details and impacted files
@@           Coverage Diff            @@
##           master    #9868    +/-   ##
========================================
  Coverage   84.18%   84.18%            
========================================
  Files         648      648            
  Lines       39668    39668            
  Branches     3541     3469    -72     
========================================
  Hits        33395    33395            
- Misses       5886     6127   +241     
+ Partials      387      146   -241     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@camd
camd force-pushed the camd/github-actions-ci branch from ed3b697 to 901d2fe Compare September 15, 2026 16:43
@camd camd self-assigned this Sep 15, 2026
@camd
camd force-pushed the camd/github-actions-ci branch 2 times, most recently from 5517e84 to 79392b3 Compare September 22, 2026 16:28
@camd
camd force-pushed the camd/github-actions-ci branch 3 times, most recently from 27c350a to 03b59dc Compare October 6, 2026 17:23
@camd
camd force-pushed the camd/github-actions-ci branch 2 times, most recently from f9d4a03 to 510d6ed Compare October 11, 2026 17:57
First step of migrating CI from CircleCI to GitHub Actions. This adds a
`ci.yml` workflow that mirrors every CircleCI job except `deploy`:

- javascript-tests, javascript-integration-tests, javascript-lint
- builds (docs + linters via tox)
- python-tests-{general,perf,frontend,telemetry} (tox docker envs)
- test-docker-build (BuildKit build of docker/Dockerfile, no push)

Job names match the CircleCI job names so required status checks can be
swapped one-for-one at cutover. A composite `setup-pnpm` action replaces
the CircleCI `setup-pnpm` command. Dependabot now also tracks action
versions.

CircleCI keeps running unchanged; it is removed in a later PR once the
two systems have run side by side and the deploy workflow is in place.
@camd
camd force-pushed the camd/github-actions-ci branch from 510d6ed to 2f76ca7 Compare October 13, 2026 16:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants