Repository navigation
ci: review every pull request with cubic, and risky ones with CodeRabbit #773
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
b57d78b
ci: review every pull request with cubic, and risky ones with CodeRabbit
Seb-GRAF dadc114
ci: ask CodeRabbit again every hour when its review limit held a revi…
Seb-GRAF efa7549
ci: retry CodeRabbit per latest commit, and label pull requests that …
Seb-GRAF 1b3cd1d
ci: keep cubic from fixing pull requests on its own
Seb-GRAF 5b7f4d4
ci: retry CodeRabbit on its limit notice, oldest first, and list the …
Seb-GRAF 9ad1ec5
ci: label pull requests that change how package sources are fetched
Seb-GRAF File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| Sources/StableBar/PrivateAPI.swift | ||
| Sources/StableBar/AppUpdate.swift | ||
| Sources/StableBar/UpdateRelease.swift | ||
| Sources/StableBar/ICloudSync*.swift | ||
| Sources/StableBar/Uninstall*.swift | ||
| Sources/StableBar/ConfigurationLoader.swift | ||
| Sources/StableBar/PackageSecrets.swift | ||
|
Seb-GRAF marked this conversation as resolved.
Resolved
|
||
| Sources/StableBar/PackageSourceFetching.swift | ||
| Resources/Info.plist | ||
| Resources/StableBar.entitlements | ||
| .github/workflows/** | ||
|
Seb-GRAF marked this conversation as resolved.
Resolved
|
||
| .github/scripts/** | ||
| .github/deep-review-paths | ||
| .github/CODEOWNERS | ||
| .github/dependabot.yml | ||
| .coderabbit.yaml | ||
| cubic.yaml | ||
| .githooks/** | ||
| scripts/** | ||
| .releaserc.json | ||
| Package.swift | ||
| Package.resolved | ||
| package.json | ||
| package-lock.json | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| // Decides whether a pull request gets the deep-review label, which starts a CodeRabbit review. | ||
| // .github/workflows/deep-review.yml runs the default branch's copy of this module, so a pull request | ||
| // never runs its own. | ||
| import { readFileSync } from "node:fs"; | ||
| import { matchesGlob } from "node:path"; | ||
|
|
||
| export const deepReviewPaths = readFileSync(new URL("../deep-review-paths", import.meta.url), "utf8").trim().split("\n"); | ||
|
|
||
| export function needsDeepReview(files, changedLines) { | ||
| return changedLines > 400 || files.some((file) => deepReviewPaths.some((path) => matchesGlob(file, path))); | ||
| } | ||
|
|
||
| // Whether to add the label anew, which asks CodeRabbit again: at least 30 minutes ago, its summary | ||
| // comment said it hit the review limit on the head commit. CodeRabbit doesn't retry by itself, and | ||
| // drops the notice once it reviews. | ||
| export function needsRetry({ headSha, comments, now }) { | ||
| return comments.some((comment) => comment.user.login === "coderabbitai[bot]" | ||
| && comment.body.includes("rate limited by coderabbit.ai") && comment.body.includes(headSha) | ||
| && now - Date.parse(comment.updated_at) >= 30 * 60 * 1000); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| import { test } from "node:test"; | ||
| import assert from "node:assert/strict"; | ||
| import { execFileSync } from "node:child_process"; | ||
| import { matchesGlob } from "node:path"; | ||
| import { deepReviewPaths, needsDeepReview, needsRetry } from "./deep-review.mjs"; | ||
|
|
||
| test("needsDeepReview asks for more than 400 changed lines", () => { | ||
| assert.equal(needsDeepReview(["docs/widgets.md"], 400), false); | ||
| assert.equal(needsDeepReview(["docs/widgets.md"], 401), true); | ||
| }); | ||
|
|
||
| test("needsDeepReview matches the listed paths", () => { | ||
| for (const file of ["Sources/StableBar/PrivateAPI.swift", "Sources/StableBar/ICloudSyncSettings.swift", | ||
| ".github/workflows/ci.yml", ".github/scripts/fixtures/screens.json", "scripts/build.sh", "Package.resolved"]) | ||
| assert.equal(needsDeepReview(["docs/widgets.md", file], 10), true, file); | ||
| for (const file of ["Sources/StableBar/BarStore.swift", "Sources/StableBar/Settings/PrivateAPI.swift", | ||
| "docs/scripts.md", "Tests/StableBarTests/UninstallTests.swift"]) | ||
| assert.equal(needsDeepReview([file], 10), false, file); | ||
| }); | ||
|
|
||
| test("every listed path matches a file", () => { | ||
| const files = execFileSync("git", ["ls-files"], | ||
| { cwd: new URL("../..", import.meta.url), encoding: "utf8", timeout: 60_000 }).split("\n"); | ||
| for (const path of deepReviewPaths) assert.ok(files.some((file) => matchesGlob(file, path)), path); | ||
| }); | ||
|
|
||
| test("needsRetry asks CodeRabbit again 30 minutes after its review limit held back the head commit", () => { | ||
| const limited = (user, head) => ({ user: { login: user }, updated_at: "2026-11-17T09:00:00Z", | ||
| body: `<!-- This is an auto-generated comment: rate limited by coderabbit.ai -->\nbetween a1 and ${head}.` }); | ||
| const summary = { user: { login: "coderabbitai[bot]" }, updated_at: "2026-11-17T09:00:00Z", | ||
| body: "<!-- This is an auto-generated comment: summarize by coderabbit.ai -->\nbetween a1 and b2." }; | ||
| const pr = { headSha: "b2", comments: [limited("coderabbitai[bot]", "b2")], now: Date.parse("2026-11-17T09:30:00Z") }; | ||
| assert.equal(needsRetry(pr), true); | ||
| assert.equal(needsRetry({ ...pr, now: Date.parse("2026-11-17T09:29:00Z") }), false); | ||
| assert.equal(needsRetry({ ...pr, headSha: "c3" }), false); | ||
| assert.equal(needsRetry({ ...pr, comments: [summary] }), false); | ||
| assert.equal(needsRetry({ ...pr, comments: [limited("cubic-dev-ai[bot]", "b2")] }), false); | ||
| assert.equal(needsRetry({ ...pr, comments: [] }), false); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| name: Deep review | ||
|
|
||
| # Adds the deep-review label, which starts a CodeRabbit review, and every hour asks CodeRabbit again | ||
| # where its review limit held a review back. pull_request_target runs the default branch's copy of | ||
| # this workflow and of .github/scripts, so a pull request never runs its own code here. The logic and | ||
| # its tests are in .github/scripts/deep-review.mjs. | ||
| on: | ||
| pull_request_target: | ||
| types: [opened, reopened, synchronize, ready_for_review] | ||
| schedule: | ||
| - cron: "17 * * * *" | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
|
Seb-GRAF marked this conversation as resolved.
Resolved
|
||
|
|
||
| concurrency: | ||
| group: deep-review-${{ github.event.pull_request.number }} | ||
| # A retry run that stopped between removing and adding the label would leave the label off. | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request_target' }} | ||
|
|
||
| jobs: | ||
| label: | ||
| name: Deep review label | ||
| if: >- | ||
|
M4n0x marked this conversation as resolved.
Resolved
|
||
| github.event_name == 'pull_request_target' && !github.event.pull_request.draft | ||
| && !contains(github.event.pull_request.labels.*.name, 'deep-review') | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| with: | ||
| ref: ${{ github.event.repository.default_branch }} | ||
| sparse-checkout: .github/scripts | ||
| persist-credentials: false | ||
| - uses: actions/github-script@v9 | ||
| with: | ||
| script: | | ||
| const { needsDeepReview } = await import(`${process.env.GITHUB_WORKSPACE}/.github/scripts/deep-review.mjs`); | ||
| const { owner, repo } = context.repo; | ||
| const pr = context.payload.pull_request; | ||
| const files = await github.paginate(github.rest.pulls.listFiles, { owner, repo, pull_number: pr.number, per_page: 100 }); | ||
| const paths = files.flatMap((file) => file.previous_filename ? [file.filename, file.previous_filename] : [file.filename]); | ||
| if (needsDeepReview(paths, pr.additions + pr.deletions)) | ||
| await github.rest.issues.addLabels({ owner, repo, issue_number: pr.number, labels: ["deep-review"] }); | ||
|
|
||
| retry: | ||
| name: Ask CodeRabbit again | ||
| if: github.event_name != 'pull_request_target' | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| with: | ||
| ref: ${{ github.event.repository.default_branch }} | ||
| sparse-checkout: .github/scripts | ||
| persist-credentials: false | ||
| - uses: actions/github-script@v9 | ||
| with: | ||
| script: | | ||
| const { needsRetry } = await import(`${process.env.GITHUB_WORKSPACE}/.github/scripts/deep-review.mjs`); | ||
| const { owner, repo } = context.repo; | ||
| const pulls = await github.paginate(github.rest.pulls.list, { owner, repo, state: "open", sort: "created", direction: "asc", per_page: 100 }); | ||
| // CodeRabbit allows each pull request author two reviews an hour; the oldest go first. | ||
| const asked = new Map(); | ||
| for (const pr of pulls.filter((pr) => !pr.draft && pr.labels.some((label) => label.name === "deep-review"))) { | ||
| if (asked.get(pr.user.login) === 2) continue; | ||
| const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number: pr.number, per_page: 100 }); | ||
| if (!needsRetry({ headSha: pr.head.sha, comments, now: Date.now() })) continue; | ||
| asked.set(pr.user.login, (asked.get(pr.user.login) ?? 0) + 1); | ||
| core.info(`Asking CodeRabbit again on #${pr.number}.`); | ||
| try { | ||
| await github.rest.issues.removeLabel({ owner, repo, issue_number: pr.number, name: "deep-review" }); | ||
| } finally { | ||
| await github.rest.issues.addLabels({ owner, repo, issue_number: pr.number, labels: ["deep-review"] }); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.