From fa6c234c3ec61f311234c2d0698deb81bbc89b68 Mon Sep 17 00:00:00 2001 From: Adam Daley Date: Fri, 9 Oct 2026 03:39:34 +0100 Subject: [PATCH 1/6] Refresh service and operations documentation Document the Stats service, current bindings and maintenance configuration, and updated migration, testing, and deployment workflows. Clarify that Extensions v2 is authoritative and update Central Alerts migration guidance. --- AGENTS.md | 56 ++++++------------------ README.md | 42 ++++++++++++++---- src/services/central-alerts/v1/README.md | 15 ++----- 3 files changed, 50 insertions(+), 63 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 5c78f03..753fd88 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,34 +10,20 @@ ### Service layout -Small services are flat: `index.ts`, `interfaces.ts`, optionally `database.ts` -and `db/` (see `central-alerts/v1`, `versions/v1`, `stats/v1`). +Small services are flat: `index.ts`, `interfaces.ts`, optionally `database.ts` and `db/` (see `central-alerts/v1`, `versions/v1`, `stats/v1`). -`extensions/v2` is the reference layout for anything larger, and new services -should grow into it rather than inventing a third shape: +`extensions/v2` is the reference layout for anything larger, and new services should grow into it rather than inventing a third shape: -- `index.ts` — app assembly only: middleware, route registration, OpenAPI - document. Route registration order is load-bearing where static paths must - beat parameter paths; those cases carry comments. +- `index.ts` — app assembly only: middleware, route registration, OpenAPI document. Route registration order is load-bearing where static paths must beat parameter paths; those cases carry comments. - `middleware.ts` — service-specific Hono middleware. -- `routes/` — one module per route group, each exporting `register*Routes(app)`. - `routes/errors.ts` maps domain error codes to HTTP status; `routes/app.ts` - holds the typed app alias. -- `db/` — `schema.ts`, `migrations/`, one `*Database` class per workflow, plus - `errors.ts` (D1 constraint classification) and `batch.ts`. -- `schemas/` — zod/OpenAPI contract split by domain. There is deliberately **no - barrel**: import from `schemas/` directly so a module's dependencies - are visible. This is why `extensions/v2` has no `interfaces.ts`. +- `routes/` — one module per route group, each exporting `register*Routes(app)`. `routes/errors.ts` maps domain error codes to HTTP status; `routes/app.ts` holds the typed app alias. +- `db/` — `schema.ts`, `migrations/`, one `*Database` class per workflow, plus `errors.ts` (D1 constraint classification) and `batch.ts`. +- `schemas/` — zod/OpenAPI contract split by domain. There is deliberately **no barrel**: import from `schemas/` directly so a module's dependencies are visible. This is why `extensions/v2` has no `interfaces.ts`. - `github/` — outbound GitHub calls, kept out of the persistence modules. -Route modules import `getExtensionsDb`/`getAuth`/`getPlatform` and middleware -directly. There is no dependency-injection container; tests drive the real app -through `app.request`. +Route modules import `getExtensionsDb`/`getAuth`/`getPlatform` and middleware directly. There is no dependency-injection container; tests drive the real app through `app.request`. -Each service documents its own contract and operational detail in its own -`README.md` (`src/services///README.md`). Keep API behaviour -there rather than here or in the root README: this file is for conventions that -apply when modifying the code. +Each service documents its own contract and operational detail in its own `README.md` (`src/services///README.md`). Keep API behaviour there rather than here or in the root README: this file is for conventions that apply when modifying the code. ## Build, Test, and Development Commands @@ -74,27 +60,11 @@ apply when modifying the code. - Local secrets go in `.dev.vars` (for example `GITHUB_TOKEN="..."`). - Bindings for D1/KV are defined in `wrangler.jsonc`; keep names aligned with `CloudflareBindings`. - Use Wrangler secrets for production tokens instead of committing them. -- `ASSERTION_SIGNING_SECRET`: HMAC key the extensions v2 API uses to verify short-lived - bearer assertions minted by the extensions site (`src/lib/auth/bearer-assertion.ts`). - Not sent over the wire — only signs/verifies server-side in each Worker. Add - `ASSERTION_SIGNING_SECRET="..."` to `.dev.vars` for local dev; set via - `wrangler secret put ASSERTION_SIGNING_SECRET` in production, matching the value - configured in the extensions site's Worker. -- `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the - extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service - binding) to purge the site's CDN-cached catalogue pages after content mutations. - Must match the value configured in the extensions site's Worker. Same local/prod - setup as `ASSERTION_SIGNING_SECRET`. +- `ASSERTION_SIGNING_SECRET`: HMAC key the extensions v2 API uses to verify short-lived bearer assertions minted by the extensions site (`src/lib/auth/bearer-assertion.ts`). Not sent over the wire — only signs/verifies server-side in each Worker. Add `ASSERTION_SIGNING_SECRET="..."` to `.dev.vars` for local dev; set via `wrangler secret put ASSERTION_SIGNING_SECRET` in production, matching the value configured in the extensions site's Worker. +- `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached catalogue pages after content mutations. Must match the value configured in the extensions site's Worker. Same local/prod setup as `ASSERTION_SIGNING_SECRET`. ## Cache Revalidation -Any endpoint that mutates catalogue-visible content (revision approve/reject, -delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, -profile deletion via `DELETE /developers/me`, claim approve/reject, extension -withdraw) must call `revalidateCatalogue(c)` from -`src/services/extensions/v2/revalidate.ts` after a successful write. Skipping it -does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows -bound staleness — but changes then take minutes instead of seconds to appear. -Profile deletion can remove an approved, extension-less profile, which is still -public content; the purge is a no-op today while no cached route carries the -`developers` tag, but keeps the endpoint correct if that ever changes. +Any endpoint that mutates catalogue-visible content (revision approve/reject, delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, profile deletion via `DELETE /developers/me`, claim approve/reject, extension withdraw) must call `revalidateCatalogue(c)` from `src/services/extensions/v2/revalidate.ts` after a successful write. + +Skipping it does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows bound staleness — but changes then take minutes instead of seconds to appear. Profile deletion can remove an approved, extension-less profile, which is still public content; the purge is a no-op today while no cached route carries the `developers` tag, but keeps the endpoint correct if that ever changes. diff --git a/README.md b/README.md index f7b1817..b4c4ea2 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,21 @@ # FOSSBilling API Worker -This is the API service that acts as the central hub for FOSSBilling instances. It handles version checks, update information, and broadcasts system-wide alerts. +This is the API service that acts as the central hub for FOSSBilling instances. It serves version checks, system-wide alerts, the extensions catalogue, preview builds, and release stats. Everything is built on [Hono](https://hono.dev), making it lightweight and fast. While we currently deploy this to Cloudflare Workers, the code is designed to be platform-agnostic. ## What it does -The worker exposes four main services: +The worker exposes five services: -- **Versions Service** (`/versions/v1`) +- **Versions** (`/versions/v1`) The source of truth for FOSSBilling updates. It fetches release data from GitHub, caches it for performance, and helps instances decide if they need to update. - **Central Alerts** (`/central-alerts/v1`) Allows the project to push critical notifications to all FOSSBilling installations—useful for security hotfixes or major announcements. - **Extensions** (`/extensions/v1`, `/extensions/v2`) - Owns the complete Extensions domain and its `DB_EXTENSIONS` schema. The separate Extensions site keeps OIDC/session state but accesses this domain through the + Owns the complete Extensions domain and its `DB_EXTENSIONS` schema. `v2` is authoritative; `v1` is legacy read-only. The separate Extensions site keeps OIDC/session state but accesses this domain through the generated HTTPS API client; it must not bind or migrate `DB_EXTENSIONS`. See [`src/services/extensions/v2/README.md`](src/services/extensions/v2/README.md). @@ -23,6 +23,10 @@ The worker exposes four main services: Resolves FOSSBilling preview builds — the current main preview and per-PR/per-commit builds produced by FOSSBilling/FOSSBilling's GitHub Actions workflows. Read-only; GitHub Actions and R2 are the sources of truth, not this service. See [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md). +- **Stats** (`/stats/v1`) + Aggregates release data from the versions service into charts and JSON. No separate GitHub calls. + See [`src/services/stats/v1/README.md`](src/services/stats/v1/README.md). + ## Architecture We've structured the app to separate the core logic from the specific runtime environment (Cloudflare, Node, etc.). @@ -35,6 +39,8 @@ We've structured the app to separate the core logic from the specific runtime en - `src/lib/adapters/cloudflare`: Real implementations using KV and D1. - `src/lib/adapters/node`: Reference implementations (useful for testing or alternative deployments). +Cron and Durable Object code lives with its owning service and is wired in `src/app/index.ts`. + ## APIs Each service documents its own endpoints and behaviour: @@ -62,7 +68,11 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d Migrations are owned by extensions v2 and applied only from this repository — see [its README](src/services/extensions/v2/README.md#database) for the migration and adoption procedure. - **KV Namespace** (`CACHE_KV`): Caches GitHub API responses so we don't hit rate limits. - **KV Namespace** (`AUTH_KV`): Stores the `UPDATE_TOKEN` value for `/versions/v1/update`. -- **R2 Bucket** (`DOWNLOAD_BUCKET`): Backs `/previews/v1/main` — see [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md) and the comment in `wrangler.jsonc` for which bucket this points at and why. +- **R2 Bucket** (`DOWNLOAD_BUCKET`): Shared by versions, previews, and stats for release/preview zips — see [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md) and the comment in `wrangler.jsonc` for which bucket this points at and why. +- **Durable Object** (`PREVIEW_GITHUB_BUDGET`): Token bucket limiting previews GitHub calls. Policy lives in `src/services/previews/v1/budget.ts`. +- **Rate limiters** (`EXTENSION_WRITE_RATE_LIMITER`, `PROFILE_CREATION_RATE_LIMITER`): Edge pacing for extension writes and profile creation. Durable quotas are enforced in D1; see [extensions v2 README](src/services/extensions/v2/README.md#extension-resource-admission-and-retention). +- **Service binding** (`EXTENSIONS_FRONTEND`): Private binding to the Extensions site worker, used to purge cached catalogue pages after mutations. +- **Cron** (`0 * * * *`): Hourly extension resource maintenance. See [extensions v2 README](src/services/extensions/v2/README.md#maintenance-and-monitoring). ### Environment Variables @@ -75,6 +85,7 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d - `EXTENSIONS_V2_EMAIL_REPLY_TO`: Optional Reply-To address for a monitored inbox. Omit when unset (no Reply-To header is sent) — notification emails do not invite replies. - `EXTENSIONS_V2_MXROUTE_SERVER`, `EXTENSIONS_V2_MXROUTE_USERNAME`, `EXTENSIONS_V2_MXROUTE_PASSWORD`: Mailbox credentials for the MXroute SMTP API (`https://smtpapi.mxroute.com/`). Required when `EXTENSIONS_V2_EMAIL_PROVIDER=mxroute`. - `EXTENSIONS_V2_RESEND_API_KEY`: API key for Resend. Required when `EXTENSIONS_V2_EMAIL_PROVIDER=resend`. +- `EXTENSIONS_RETENTION_MODE`: `dry-run` (report only) or `compact` (prune old revision bodies). See [extensions v2 README](src/services/extensions/v2/README.md#maintenance-and-monitoring). Only extensions v2 consumes these. For the assertion format and the rotation procedure, see [its README](src/services/extensions/v2/README.md#authentication). @@ -101,8 +112,7 @@ npm install 2. Apply migrations to the local D1 databases: ```bash - npm run db:migrate:extensions-v2:local - npm run db:migrate:central-alerts:local + npm run db:migrate:all:local ``` 3. (Optional) Store an update token in KV for `/versions/v1/update`: @@ -121,8 +131,22 @@ You can now hit endpoints at `http://localhost:8787`. ### Testing -We use Vitest for testing. The suite includes unit tests for the endpoints and integration tests using the platform adapters. +We use Vitest. Unit tests run under the Workers pool; Node adapter tests use a separate config. + +```bash +npm run test # workers pool +npm run test:all # workers + node adapters +``` + +```bash +npm run typecheck +npm run lint +``` + +### Deploy ```bash -npm run test +npm run cf-deploy ``` + +This applies remote D1 migrations, then deploys with Wrangler. diff --git a/src/services/central-alerts/v1/README.md b/src/services/central-alerts/v1/README.md index a6cf5fe..1720de4 100644 --- a/src/services/central-alerts/v1/README.md +++ b/src/services/central-alerts/v1/README.md @@ -8,18 +8,11 @@ The Central Alerts service provides targeted notifications to FOSSBilling instal ### GET `/list` -Retrieve all alerts in the system. Optional `limit` (integer 1–100) and -`offset` enable pagination. An unusable limit keeps the full-list response; -`offset` without a usable limit returns 422. An omitted or unusable offset -with a usable limit defaults to zero. +Retrieve all alerts in the system. Optional `limit` (integer 1–100) and `offset` enable pagination. An unusable limit keeps the full-list response; `offset` without a usable limit returns 422. An omitted or unusable offset with a usable limit defaults to zero. -Successful responses are edge-cached for 60 seconds by the effective page. -Unknown query parameters and equivalent pagination spellings reuse the same -entry. This endpoint is public: Authorization does not affect its response or -bypass its cache. Validation failures and database errors are not cached. +Successful responses are edge-cached for 60 seconds by the effective page. Unknown query parameters and equivalent pagination spellings reuse the same entry. This endpoint is public: Authorization does not affect its response or bypass its cache. Validation failures and database errors are not cached. -When `limit` is usable the response adds a `pagination` object next to -`alerts`: +When `limit` is usable the response adds a `pagination` object next to `alerts`: ```json { @@ -79,4 +72,4 @@ The `type` field accepts: `success`, `info`, `warning`, `danger` ## Database -Uses D1 database binding `DB_CENTRAL_ALERTS`. Initialize with the setup script in `src/services/central-alerts/v1/scripts/`. +Uses D1 database binding `DB_CENTRAL_ALERTS`. Apply migrations from `db/migrations` with `npm run db:migrate:central-alerts:local` / `:remote`. From dbc9b47079de77b1a1f9eb4bca60fdb59fa5a02f Mon Sep 17 00:00:00 2001 From: Adam Daley Date: Fri, 9 Oct 2026 03:44:15 +0100 Subject: [PATCH 2/6] Condense extensions v2 service documentation Reorganize the service README into shorter sections covering lifecycle, reads, authentication, limits, retention, and database migrations while preserving key API and operational guidance. --- src/services/extensions/v2/README.md | 481 +++++---------------------- 1 file changed, 91 insertions(+), 390 deletions(-) diff --git a/src/services/extensions/v2/README.md b/src/services/extensions/v2/README.md index c0eab25..dc7a7a1 100644 --- a/src/services/extensions/v2/README.md +++ b/src/services/extensions/v2/README.md @@ -4,7 +4,7 @@ Self-service extension publishing, developer-profile ownership, moderation, and public catalogue browsing. -This service owns the complete Extensions domain and its `DB_EXTENSIONS` schema: users, developers, submissions, claims, transfers, history, and catalogue data. The separate Extensions site keeps OIDC/session state but reaches this domain through the generated HTTPS API client; it must not bind or migrate `DB_EXTENSIONS`. +This service owns the complete Extensions domain and its `DB_EXTENSIONS` schema. The separate Extensions site keeps OIDC/session state but reaches this domain through the generated HTTPS API client; it must not bind or migrate `DB_EXTENSIONS`. ## Endpoints @@ -13,319 +13,84 @@ Endpoints are not listed here. The service publishes its own contract: - **OpenAPI document:** `GET /extensions/v2/docs/openapi.json` - **Reference UI:** `GET /extensions/v2/docs` -## The Extension Lifecycle - -An extension is a resource from the moment a developer creates it, not from the -moment a moderator approves it. There is no separate "submission" to reconcile -against it. - -- `POST /extensions` creates the record. It holds its id immediately and stays - out of both catalogues until its first revision is approved. -- `PUT /extensions/{id}` proposes an edit. The published content does not - change until a moderator approves the revision, and only one revision per - extension may be unreviewed at a time. -- `DELETE /extensions/{id}` withdraws an extension that has never been - published, releasing its id. A published extension cannot be withdrawn by its - owner — consumers pin the id. -- `POST /extensions/{id}/revisions/{revisionId}/approve` publishes the - revision's content. `reject` leaves the published content untouched and the - extension available to edit and resubmit. -- `POST /extensions/{id}/delist` pulls an already-published extension out of - the public catalogue for cause (its upstream source disappearing, for - example). Moderator-only, and the inverse of neither `approve` nor - `reject`: content and history are kept, so the owner can still see and edit - the extension. `POST /extensions/{id}/relist` restores it (optional - `review_note`, same `?notify` opt-out); see `ExtensionsDatabase.delist()` - and `relist()`. -- `POST /extensions/{id}/moderator-correct` corrects a published extension's - live content as a moderator (api#251): truncated readmes, broken links, or - other corruption that should not wait for the author to resubmit. One write - inserts an already-approved revision row (`submitted_by` and `reviewer_id` - both the moderator, `correction_note` as the `review_note`) and publishes - it, leaving `published_at` and ownership untouched; the catalogue is - revalidated afterwards. Requires a published, listed extension with no - pending revision (409 otherwise — approve or reject the pending edit - first, never supersede it). Unlike every other moderation write there is no - `?notify` query and no author email: the correction is recorded in revision - history and surfaced by the directory UI. See `ExtensionsDatabase.moderatorCorrect()`. -- `GET /extensions/{id}` is role-aware: anonymous and unrelated callers get - the published projection (or 404, which hides existence for drafts and - delisted rows); the owner and moderators get the full `OwnedExtension`, - including `delisted`. This replaces the former `GET /extensions/mine/{id}` - and `GET /moderation/extensions/{id}` pair, which shared that shape. -- `GET /extensions?scope=` lists catalogue cards or owned rows: - `scope=public` (default, anonymous allowed) is the published catalogue; - `scope=mine` is the caller's own extensions (empty page when they have no - developer profile); `scope=all` (moderator only) is every extension, - filterable by `status` (`published`, `delisted`, `unpublished`) and `q` - (case-insensitive id substring). Replaces `GET /extensions/mine` and - `GET /moderation/all-extensions`. An extension's own status and its having - a pending edit are independent. - -### Moderation Notification Emails - -Revision approve/reject, delist/relist, developer approve, and claim -approve/reject email the affected author unless the moderator opts out with -`?notify=false`. Automatic decisions by the FOSSBilling Bot account use the -same routes and mail path; they are identified by the `[auto policy=…]` -`review_note` prefix (no schema change). -The recipient is the developer's `contact_email`, falling back to the owning -account's `email`; claim decisions go to the claimant's account email. -Sending is best-effort and never fails the write: the result carries -`notified: boolean`, which reports whether a recipient was resolved and the -send dispatched. The provider POST itself runs via `waitUntil` after the -response (it has a 10s abort timeout and is not worth blocking a moderation -write on), so delivery failures surface only in logs. A missing address or -missing mail credentials skip the send before the response and report -`notified: false`. See `email/` for the provider abstraction (`mxroute` via -`https://smtpapi.mxroute.com/`, `resend`, or `disabled`) and the root README -for the `EXTENSIONS_V2_EMAIL_*` configuration. - -### Correcting Live Content - -Three paths, in order of preference (api#251): - -- **Moderator edit** (`POST /extensions/{id}/moderator-correct`): live - catalogue corruption — a truncated readme, a broken link — on a published, - listed extension. Immediate, audited (an approved revision row), no author - email. -- **Ask the author to resubmit**: wording disputes, new releases, unpublished - or delisted extensions, or anything while a pending edit exists. The normal - propose/review queue, with its notification emails. -- **Direct D1 edit (break-glass only)**: the API is down, or the stored data - violates a constraint the API cannot express a fix through. Back up the row - first, write down why in the change record, and trigger a catalogue - revalidate manually afterwards — the audit trail and purge that the API - would have done do not happen by themselves. - -The id and the developer are properties of the extension, not of a revision: an -edit cannot rename an extension or move it to another developer, and approving -one no longer rewrites the developer profile as a side effect. A user owns at -most one developer profile, so no request body names one. - -### Reading Owner State - -`GET /extensions?scope=mine` and `GET /extensions/{id}` (as owner/moderator) -return four independent fields rather than a single derived status, because -together they are the state and a derived enum could only disagree with them. The table below -covers three of them - `published`, `pending_revision` and `last_review` - the -fourth, `delisted`, is documented separately just below since it is orthogonal -to all three: - -| `published` | `pending_revision` | `last_review` | Meaning | -| ----------- | ------------------ | ------------- | --------------------------------------- | -| `null` | set | `null` | Awaiting first review | -| `null` | set | rejected | Rejected, and already resubmitted | -| `null` | `null` | rejected | Rejected; edit and resubmit | -| set | `null` | approved | Live, no unreviewed edit | -| set | `null` | `null` | Live, adopted from the pre-v2 catalogue | -| set | set | either | Live, with an edit awaiting review | - -The adopted row is the one worth reading twice: migration 0021 published every -extension that already existed, and those have no revisions at all, so a live -extension with no review history is normal rather than a gap. `published` -being set is the only thing that means "in the catalogue" - except a fourth, -independent field, `delisted`: set once a moderator removes a published -extension for cause, it hides the row from both public catalogue reads -without touching `published`, `pending_revision` or `last_review`. - -Merged reads return a union narrowed by the request: `?scope=` selects the -list projection explicitly, while `GET /extensions/{id}` and -`GET /developers/{id}` return the published/public shape anonymously and the -owned/full shape for the owner or a moderator (for extensions, 404 otherwise -hides draft and delisted rows). -Anonymous reads stay cacheable (`Cache-Control: public`); authenticated reads -send `Vary: Authorization`. - -`GET /extensions/{id}/revisions` lists the full history of one extension, -newest first, for its owner or any moderator. `GET /revisions` is the global -review queue (moderator only, `?status=` defaulting to `pending`, oldest -first). - -`GET /developers?scope=` (`all` default, `unapproved` for the review queue; -`?status=` remains as a deprecated alias during the coordinated migration and -422s when it disagrees with `?scope=`) replaces `GET /developers/unapproved`. -`GET /developers/claims?scope=mine` (the caller's claims) and `?scope=pending` -(moderator queue) replace `GET /developers/claims/mine` and -`GET /developers/claims`; both return the enriched pending shape, and a -`status` filter disagreeing with `scope=pending` is rejected with 422 rather -than silently ignored. `GET /developers/{id}` is role-aware like -`GET /extensions/{id}`: public view anonymously, full view for the owner or a -moderator. `PATCH /users/me` returns the full account projection, like -`GET /users/me`. - -`GET /developers`, `GET /developers/claims`, and `GET /developers/{id}/history` -page by opaque keyset cursor like every other v2 list: `?limit=` (1-100, -default 50) with `?cursor=` carried from the previous page's -`pagination.next_cursor`. An invalid cursor is rejected with `INVALID_CURSOR` -(422); the response envelope is always `pagination: {next_cursor, has_more}`. +## Lifecycle -## Authentication +An extension exists from `POST /extensions`, not from approval. There is no separate submission record. + +- `POST /extensions` creates the record and reserves its id. It stays out of catalogues until its first revision is approved. +- `PUT /extensions/{id}` proposes an edit. Published content is unchanged until approval. One unreviewed revision per extension at a time. +- `DELETE /extensions/{id}` withdraws an unpublished extension and releases its id. Published extensions cannot be withdrawn by owners. +- `POST /extensions/{id}/revisions/{revisionId}/approve` publishes. `reject` leaves published content untouched. +- `POST /extensions/{id}/delist` (moderator-only) hides a published extension for cause without deleting content or history. `POST /extensions/{id}/relist` restores it. Both accept `?notify=false`. +- `POST /extensions/{id}/moderator-correct` (moderator-only) fixes live catalogue corruption without waiting for the author. Writes an already-approved revision, leaves `published_at` and ownership untouched. Requires a published, listed extension with no pending revision (409 otherwise). No `?notify` option and no author email; the fix is recorded in revision history. + +An edit cannot rename an extension or move it to another developer. A user owns at most one developer profile, so no request body names one. + +## Reads + +`GET /extensions/{id}` and `GET /developers/{id}` are role-aware: anonymous callers get the published projection (404 hides drafts and delisted rows); owners and moderators get the full object. + +`GET /extensions?scope=` selects the projection: + +- `public` (default, anonymous allowed): published catalogue cards. +- `mine`: caller's own extensions. +- `all` (moderator only): every extension, filterable by `status` (`published`, `delisted`, `unpublished`) and `q` (case-insensitive id substring). + +Owner/moderator extension reads return four independent fields, not a derived status: -Requests carry a short-lived bearer assertion minted by the Extensions site and verified here with a shared HMAC secret (`ASSERTION_SIGNING_SECRET`; see the root README for where to configure it). +| `published` | `pending_revision` | `last_review` | Meaning | +| ----------- | ------------------ | ------------- | ----------------------------------- | +| `null` | set | `null` | Awaiting first review | +| `null` | set | rejected | Rejected, already resubmitted | +| `null` | `null` | rejected | Rejected; edit and resubmit | +| set | `null` | approved | Live, no unreviewed edit | +| set | `null` | `null` | Live, adopted from pre-v2 catalogue | +| set | set | either | Live, edit awaiting review | + +`delisted` is orthogonal to all three: it hides the row from public reads without touching them. Adopted rows have no revisions; a live extension with no review history is normal. + +`GET /extensions/{id}/revisions` lists one extension's history (owner/moderator, newest first). `GET /revisions` is the global review queue (moderator only, `?status=` defaults to `pending`, oldest first). `GET /developers?scope=` (`all` default, `unapproved` for the queue) and `GET /developers/claims?scope=` (`mine`, `pending`) follow the same pattern. `PATCH /users/me` returns the full account projection, like `GET /users/me`. + +Anonymous reads are cacheable (`Cache-Control: public`); authenticated reads send `Vary: Authorization`. + +## Pagination + +All v2 lists use opaque keyset cursors: `?limit=` (1–100, default 50) with `?cursor=` from the previous page's `pagination.next_cursor`. Envelope is always `pagination: {next_cursor, has_more}`. An invalid cursor returns `INVALID_CURSOR` (422); restart from the first page. List items omit `readme` and `releases`; fetch `GET /extensions/{id}` for detail. + +## Authentication -Assertions use HS256 and include the exact issuer `fossbilling-extensions`, audience `fossbilling-api/extensions-v2`, purpose `user-authentication`, and protocol version `1`. They are valid for at most 60 seconds. +Requests carry a short-lived bearer assertion minted by the Extensions site, verified here with `ASSERTION_SIGNING_SECRET` (see root README). -`PUT /users/me/identity` instead requires purpose `identity-sync` with a -`body_sha256` claim containing the lowercase hexadecimal SHA-256 digest of the -exact UTF-8 JSON request bytes. Other claims and lifetime requirements stay the -same. The site must derive the projection from trusted provider data, serialize -once, hash those bytes, sign, and send the same bytes. Never mint identity proofs -for browser-supplied projections or expose these proofs to clients. -Ordinary user assertions and mismatched bodies receive 403 on this endpoint; -missing or invalid credentials receive 401. Identity proofs cannot authorize -other API routes. Schema validation remains 422 after proof verification. +Assertions use HS256 with issuer `fossbilling-extensions`, audience `fossbilling-api/extensions-v2`, purpose `user-authentication`, protocol version `1`. Valid for at most 60 seconds. -Deploy the site's new identity-sync signer with this API change; the previous -unsigned-body sync protocol is intentionally rejected. Membership expiry is -capped to one hour from synchronization and never extended past the supplied -expiry. Invalid or stale evidence remains unavailable for automatic verification. -Review historical identity projections and derived verification records if prior -abuse is suspected; this ingress fix does not attest to previously stored data. +`PUT /users/me/identity` requires purpose `identity-sync` plus a `body_sha256` claim holding the lowercase hex SHA-256 of the exact UTF-8 JSON request bytes. The site must hash the same bytes it sends. Identity proofs return 403 on other routes; missing/invalid credentials return 401; schema failures return 422. Membership expiry is capped at one hour. -### Rotating the Shared Secret +### Rotating the shared secret -`ASSERTION_SIGNING_SECRET_PREVIOUS` is an optional second secret accepted only as a temporary rotation window. To rotate without interrupting requests: +`ASSERTION_SIGNING_SECRET_PREVIOUS` is accepted only as a temporary rotation window: 1. Set the API's `ASSERTION_SIGNING_SECRET_PREVIOUS` to the current value. 2. Replace the API's active `ASSERTION_SIGNING_SECRET`. 3. Replace the Extensions site's active secret. -4. After at least 65 seconds, verify requests and remove the API's previous secret. +4. After at least 65 seconds, verify traffic and remove the previous secret. -Remove the previous secret once the new one has been active for at least 65 seconds and all in-flight assertions have expired. +## Ownership verification -## Ownership Verification +For organization developer IDs, GitHub membership auto-verifies only against a valid, unexpired membership snapshot. A fresh snapshot without the org is a confirmed mismatch. Missing, malformed, or expired evidence is inconclusive: the profile stays unapproved and claims stay pending for manual review. `github_org_verified` absent or `null` is a review signal, not proof. -For organization developer IDs, GitHub membership is used for automatic verification only when the API has a valid, unexpired membership snapshot. A fresh snapshot that does not contain the organization remains a confirmed mismatch and is rejected. Missing, malformed, or expired evidence is inconclusive instead: a new profile remains unapproved and a claim remains pending for manual moderator review. Moderators must verify ownership through their normal out-of-band process before approving either workflow. +## Notification emails -`github_org_verified` being absent or `null` is a review signal, not proof of ownership or an authorization grant. Consumers and moderation tooling must not treat an inconclusive result as verified. +Revision approve/reject, delist/relist, developer approve, and claim approve/reject email the affected author unless the moderator passes `?notify=false`. Bot decisions use the same path, marked with an `[auto policy=…]` `review_note` prefix. -## List Pagination +Recipients: developer `contact_email`, falling back to the owning account's email; claim decisions go to the claimant's account email. Sending is best-effort via `waitUntil` and never fails the write; the result carries `notified: boolean`. See `email/` for the provider abstraction (`mxroute`, `resend`, `disabled`) and the root README for `EXTENSIONS_V2_EMAIL_*`. -`GET /extensions/v2/extensions` returns bounded pages. `scope=public` (default) -returns lightweight published catalogue items, `scope=mine`/`all` return owned -rows. List items intentionally omit `readme` and `releases`; retrieve the full object from `GET /extensions/v2/extensions/{id}` for detail views. Follow `pagination.next_cursor` by passing it unchanged as `cursor`, and treat cursors as opaque. The default page size is 50 and `limit` may be set from 1 through 100. +## Limits and budgets -Cursors carry a version field and are validated on decode, so a cursor from an older format is rejected with `INVALID_CURSOR` (HTTP 422) rather than being misread. Clients should treat that as "restart pagination from the first page", not as an error to surface. +Request size is counted on the raw stream before JSON parsing. Raw limit is **512 KiB** (`413 BODY_TOO_LARGE`); normalized JSON content limit is **256 KiB**. New slug ids are at most 200 characters. -## Database +`EXTENSION_WRITE_RATE_LIMITER` paces IP and account attempts at 60/minute, including validation failures. It is approximate edge pacing, not the durable quota. Missing pacing fails closed (`503 ADMISSION_UNAVAILABLE`). -Uses the D1 binding `DB_EXTENSIONS`, shared with v1 (read-only there). This service owns the schema and the migrations. - -`extensions` holds the record; its content columns are the _published_ -projection and are NULL until a first approval, with `published_at` as the -marker both catalogues filter on and `extensions_published_content_check` -guaranteeing a published row is never half-written. `extension_revisions` -(renamed from `extension_submissions` in migration 0021) holds proposed -content, always attached to a real extension row and cascading with it. - -Migration 0021 rebuilds `extensions` and replaces `extension_submissions` with -`extension_revisions`, because SQLite cannot relax `NOT NULL`, add a `CHECK`, or -add a foreign key in place. It also renames `extensions.author_id` to -`developer_id` — nothing public depended on the old name, since v1's response -field is `author` either way. - -**Ordering in 0021 is load-bearing.** It never drops a table that still has -children, so `extension_submissions` is copied aside and dropped before -`extensions` is rebuilt. Foreign keys cannot be relaxed to avoid this: -`PRAGMA foreign_keys` is a no-op inside a transaction and wrangler wraps each -migration file in one, while `PRAGMA defer_foreign_keys` does not help either — -dropping a parent increments SQLite's deferred-violation counter per child row -and nothing decrements it, so the commit fails even when the data is sound. The -same constraint is why `developers` is not rebuilt: three tables reference it. -`migrations.test.ts` applies the chain under those conditions so this cannot -regress. - -Apply migrations **only from this repository**, from `db/migrations`, with `npm run db:migrate:extensions-v2:local` / `:remote`. The Extensions site has no D1 migration source. - -Migration `0020` is a check, not a schema change: it fails if an adopted developer row holds an id that a static route shadows (`developers.id` of `me`/`claims`/`unapproved`), which would make that row's detail page unreachable. If it fails, rename the row deliberately — the id is public and consumers pin it. - -Migration `0021` refuses to run against data it cannot migrate, rather than aborting halfway through the rebuild. Each check selects the offending rows into a scratch table whose named `CHECK` can never hold, so the constraint name is the error message — SQLite has no `RAISE()` outside a trigger: - -| Failure | Meaning | -| -------------------------------------------- | ----------------------------------------------------------------------- | -| `extension_ids_must_not_differ_only_by_case` | Two catalogue ids collide under `idx_extensions_id_nocase` | -| `submission_target_ids_must_not_be_reserved` | A submission targets an id a static route shadows | -| `extension_references_must_resolve` | The `foreign_keys=OFF` rebuild would carry a dangling reference through | - -None of these are repaired automatically: each is a decision about published data that belongs to a human. Reconcile and re-run — the migration has touched nothing at that point. - -It does resolve one case itself: pending submissions whose ownership state can never satisfy approval are rejected, since one pending revision per extension would otherwise block the owner's next edit forever. - -Migration `0021` also drops any submission filed under a developer that no longer exists: there is no `developer_id` such a row could carry that satisfies the new foreign key, and the profile it was filed under is already gone. - -## Code Layout - -See `AGENTS.md` for what belongs in `routes/`, `db/`, `schemas/`, `github/`, and `middleware.ts`. This service is the reference layout for larger services. - -### Claim verification budget - -Claim verification is limited before contacting GitHub: 3 attempts per account -and per normalized developer ID per 60 seconds, plus 300 aggregate claim -verification attempts per hour. D1 reserves all budgets atomically. Mismatches, -upstream failures, and cancellation do not refund attempts. Exhaustion returns -429 (`RATE_LIMITED`); sequential replays of a pending claim short-circuit to -409 before spending quota, but two requests racing each other can each spend -one attempt before the unique index decides the winner. Deploy migration -`0024_claim_verification_budgets.sql` before the Worker update. -The aggregate budget covers claims only and leaves shared GitHub capacity for -other workflows; it is not a budget for every GitHub consumer. - -### Developer profile approval - -Every meaningful profile edit clears manual approval, including edits by -GitHub-verified owners. Identical submissions preserve it. GitHub identity and -website verification remain separate signals. Profile reads, embedded extension -developers, and the review queue require an approval for the current revision. - -Owner and moderator reads include `profile_generation` (a 32-character lowercase -hexadecimal token) and `content_revision`; anonymous reads omit both. -`POST /developers/{id}/approve` requires -`{ "expected_generation": "", "expected_revision": 1 }`. -Send both values from the profile actually reviewed. Changed or recreated -profiles return 409; missing or invalid tokens return 422. Each creation gets a -fresh generation token, including reused developer ids. - -Apply migration `0027_bind_developer_profile_generation.sql` and deploy the API -and updated moderation client before resuming moderation. Older approval requests -are rejected. The migration clears existing manual approvals for fresh review -because the previous rules could preserve them across unreviewed edits. GitHub -verification is retained. - -### Profile write budget - -`PUT /developers/me` permits 20 successful profile writes per account in a -rolling 24-hour window, including creation. Exhaustion returns 429 with -`PROFILE_MUTATION_RATE_LIMITED` and a conservative `Retry-After: 86400`. -Unchanged submissions return the current profile without changing approval, -revision, audit history, or revalidating the catalogue. All five editable fields -participate in the comparison; omitted optional fields mean null. The budget is -enforced atomically with the write using indexed, immutable history and survives -profile deletion, recreation, and ownership transfer. Audit records remain -append-only; this bounds growth per account per day rather than total retention. -Apply migration `0025_luxuriant_franklin_richards.sql` before deploying. - -## Extension resource admission and retention - -Content writes (`POST /extensions`, `PUT /extensions/{id}`, and moderator -correction) count the actual request stream before JSON parsing. The raw request -limit is **512 KiB**, including whitespace, escapes, and unknown fields. An -oversized stream is canceled and returns `413 BODY_TOO_LARGE`; `Content-Length` -cannot bypass counting. The separate normalized JSON content limit remains -**256 KiB**, with the existing field/release maxima. New extension slug IDs are at most -200 characters. Previously stored IDs had no length cap and remain supported. - -The `EXTENSION_WRITE_RATE_LIMITER` binding paces IP and authenticated-account -attempts at 60/minute, including validation failures. This edge control is -approximate and must not be used as the durable quota. Missing/unavailable -attempt pacing fails closed with `503 ADMISSION_UNAVAILABLE`. Unavailable durable -write admission returns the same response with `Retry-After: 60`. - -Migration `0026_resource_bounds.sql` enforces the following accepted-write and -retained-resource limits **inside the write transaction**, including moderator -corrections. Its trigger guards and accounting are shared by create, edit, -approval, correction, withdrawal and retention; a failed batch leaves neither -an extension nor an accepted-write charge behind. +Migration `0026_resource_bounds.sql` enforces accepted-write and retained-resource limits inside the write transaction: | Resource | Account | Developer | | ------------------------------------- | ------- | --------- | @@ -335,99 +100,35 @@ an extension nor an accepted-write charge behind. | Extension records | 100 | 100 | | Revision metadata records | 1,000 | 1,000 | -Aggregate storage and record counts are measured for visibility; they do not -impose a whole-system admission limit. - -The pending budget remains ten revisions per submitter and one per extension. -The byte quota includes retained revision JSON **and** published content -columns. It counts UTF-8 bytes, not JavaScript string length. Original creators -are charged for extension records and published projections; revision bodies -and metadata remain charged to their submitters and original developer IDs. -Transferring a profile does not silently shift these charges to the recipient. -The `created_by` attribution is immutable. Legacy unowned publications are -charged to a synthetic `legacy` account bucket. - -Each accepted write also removes up to 50 expired ledger entries, so expiry -cleanup scales with write traffic. The hourly job handles idle cleanup. - -The accepted-write ledger has no deletion-cascading foreign keys: review, -withdrawal, profile replacement and account reactivation do not refund a -rolling allowance. Minute/day exhaustion returns `429` with a domain -code (`WRITE_RATE_MINUTE`, `WRITE_RATE_DAY`) and a conservative -`Retry-After` of 60 or 86400 seconds respectively. Retained quota exhaustion -returns `409 RESOURCE_QUOTA`. Withdrawing an unpublished extension releases its -stored bytes and record counts, but not its accepted-write allowance. - -`resource-limits.ts` mirrors the policy values used by middleware and maintenance. - -### Revision list/detail contract - -`GET /revisions` and `GET /extensions/{id}/revisions` now return -`ExtensionRevisionSummary` pages. They select stored, bounded summary fields -(`name`, `version`, `description`), decision metadata, `content_bytes`, -`content_available`, `content_hash` and `compacted_at`. They do **not** select, -transfer or parse content JSON, including the pagination lookahead row. -Timestamp/ID keyset ordering, limit 1–100 and default 50 are unchanged. - -Fetch `GET /extensions/{id}/revisions/{revisionId}` for full content on demand. -The caller must be the current owner or an active moderator, and authorization -is repeated in the actual data query. A compacted revision returns `content: -null`, `content_available: false`, a SHA-256 hash of the original stored bytes, -and its compaction timestamp. `content_bytes` measures the currently retained -body (the two-byte `{}` placeholder after compaction), not the historical body. - -Oversized legacy content remains stored and counted, but is never fetched or -parsed by list or detail handlers. Its revision detail returns `409 -CONTENT_UNAVAILABLE`. An owner/moderator extension detail with an oversized -published or pending body also returns this error. Oversized published legacy -rows are excluded from public catalogue cards. Historical release collections -are checked for safe count/tag bounds before version sorting; unsafe collections -also return `CONTENT_UNAVAILABLE`; their summaries report `content_available: false`. -The stored readability flag is calculated alongside summary metadata, including -the safe legacy release count and tag bounds (100 Unicode code points). -Anonymous public extension detail reads also return `409 CONTENT_UNAVAILABLE` -for oversized published bodies. Moderator lists still expose -the published card for correction without reading its oversized body. -Cards bound display fields and project license/repository metadata; oversized -legacy URLs are null (or an omitted optional icon), rather than clipped links. -Detail reads preserve the original fields when the body is safe to fetch. -Reject an oversized pending revision, then ask its -owner to resubmit under current bounds; use moderator correction for published -content. Administrative export is required if the original oversized body must -be recovered. The migration backfills byte counts/summaries in SQL without -loading old bodies into a Worker or discarding any rows. Existing over-quota -collections cannot grow until usage is reduced. - -### Maintenance and monitoring - -The Worker runs scheduled maintenance **hourly**, handling at most -**20 bodies** and 500 expired ledger rows per invocation. Retention defaults to -`dry-run`: it reports eligible bodies and reclaimable bytes without fetching or -changing bodies. Only `EXTENSIONS_RETENTION_MODE=compact` enables compaction; -missing or invalid mode values keep retention non-destructive. Expired admission -ledger entries and empty usage counters are cleaned in either mode. Reviewed revision -bodies older than **180 days** are compacted to `{}` while retaining their -bounded summaries, review note, reviewer, dates, status and original content -hash. Pending bodies and the currently published revision are always protected. -Oversized legacy bodies are left for administrative handling. Metadata is -retained under the count quota; reaching that quota needs deliberate export -and administrative cleanup rather than silent deletion of decisions. - -Compaction rechecks age, status, body equality and the published pointer in the -UPDATE, so concurrent review/publication changes cannot discard protected -content. Accounting is updated in the same transaction. Repeated runs are -idempotent. Expired write events and empty usage counters are cleaned in bounded -batches. - -Full resource inventory is logged after each hourly maintenance run. -Structured logs record revision-response bytes/duration, admission reason codes and maintenance/inventory totals: -`retained_bytes`, `extensions`, `revisions`, `pending`, -`oversized_legacy_revisions`, `cleanup_backlog`, `compacted`, and -`expired_events`. They contain no revision bodies or user identifiers. Warning-level signals flag -oversized legacy bodies and an active -compaction backlog of 300 or more bodies. Dry-run eligibility is informational. - -`db/resource-inventory.sql` is a manual, read-only reconciliation diagnostic, -separate from the hourly usage report. It lists at most 100 accounting -discrepancies using stored scalar sizes without returning content bodies, -ordered by descending byte/count drift with stable scope/subject tie-breaks. +Minute/day exhaustion returns `429` (`WRITE_RATE_MINUTE` / `WRITE_RATE_DAY`, `Retry-After` 60 / 86400). Retained quota exhaustion returns `409 RESOURCE_QUOTA`. Withdrawing an unpublished extension releases stored bytes and record counts, not accepted-write allowance. Pending budget: ten revisions per submitter, one per extension. See `resource-limits.ts` for the mirrored policy values. + +Claim verification is budgeted before contacting GitHub: 3 attempts per account and per developer id per 60 seconds, plus 300 aggregate attempts per hour (migration `0024`). Exhaustion returns `429 RATE_LIMITED`. Mismatches and upstream failures do not refund attempts. + +`PUT /developers/me` allows 20 successful writes per account per rolling 24 hours (migration `0025`). Exhaustion returns `429 PROFILE_MUTATION_RATE_LIMITED` with `Retry-After: 86400`. Identical submissions preserve approval and skip the write. + +Profile approval binds to the reviewed revision: owner/moderator reads include `profile_generation` and `content_revision`; `POST /developers/{id}/approve` requires both as `expected_generation` / `expected_revision` (migration `0027`). Changed profiles return 409; missing/invalid tokens return 422. Every meaningful edit clears manual approval. + +## Revisions and retention + +`GET /revisions` and `GET /extensions/{id}/revisions` return summaries (`name`, `version`, `description`, decision metadata, `content_bytes`, `content_available`, `content_hash`, `compacted_at`), never content bodies. Fetch `GET /extensions/{id}/revisions/{revisionId}` for full content (owner/moderator). A compacted revision returns `content: null` with its hash and timestamp. Oversized legacy bodies return `409 CONTENT_UNAVAILABLE`; their summaries report `content_available: false`. + +Scheduled maintenance runs hourly, handling at most 20 bodies and 500 expired ledger rows per run. Retention defaults to `dry-run` (report only); only `EXTENSIONS_RETENTION_MODE=compact` compacts reviewed bodies older than 180 days to `{}`. Pending bodies and the published revision are always protected. Inventory totals are logged after each run. `db/resource-inventory.sql` is a manual read-only reconciliation query. + +## Database + +Uses D1 binding `DB_EXTENSIONS`, shared with v1 (read-only there). This service owns the schema. Apply migrations only from this repository with `npm run db:migrate:extensions-v2:local` / `:remote`. + +- `0020`: guard check; fails if an adopted developer id shadows a static route (`me`/`claims`/`unapproved`). +- `0021`: rebuilds `extensions`, replaces `extension_submissions` with `extension_revisions`. Ordering is load-bearing (never drop a table with children); covered by `migrations.test.ts`. Refuses unmigratable data with named checks: + +| Failure | Meaning | +| -------------------------------------------- | ------------------------------------------------ | +| `extension_ids_must_not_differ_only_by_case` | Two ids collide under `idx_extensions_id_nocase` | +| `submission_target_ids_must_not_be_reserved` | A submission targets a reserved static-route id | +| `extension_references_must_resolve` | Rebuild would carry a dangling reference through | + +Reconcile and re-run; the migration touches nothing before these checks. + +## Code layout + +See `AGENTS.md` for `routes/`, `db/`, `schemas/`, `github/`, and `middleware.ts`. This service is the reference layout for larger services. From 94f6c08e153ec6a4ef58a4db2797be42d02e010d Mon Sep 17 00:00:00 2001 From: Adam Daley Date: Fri, 9 Oct 2026 03:47:59 +0100 Subject: [PATCH 3/6] Refresh AGENTS.md service layout and commands Update the repository guidelines to match the current codebase. Broaden the flat-service layout to list the narrow helper modules actually in use, and document the extensions v2 `email/` module and the `previews/v1` split. Replace the stale `deploy` script with `cf-deploy`, and add `cf-typegen`, `db:migrate:all:*` and `test:all`. List the full set of bindings in `wrangler.jsonc`. Add moderator-correct to the revalidation list and shorten the note on profile deletion. --- AGENTS.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 753fd88..2e36dd3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,7 @@ ### Service layout -Small services are flat: `index.ts`, `interfaces.ts`, optionally `database.ts` and `db/` (see `central-alerts/v1`, `versions/v1`, `stats/v1`). +Small services are flat: `index.ts` + `interfaces.ts` plus narrow helpers (`database.ts`, `db/`, `r2.ts`, `responses.ts`, `dashboard.ts`, `list-cache.ts`, depending on the service). `extensions/v2` is the reference layout for anything larger, and new services should grow into it rather than inventing a third shape: @@ -20,6 +20,7 @@ Small services are flat: `index.ts`, `interfaces.ts`, optionally `database.ts` a - `db/` — `schema.ts`, `migrations/`, one `*Database` class per workflow, plus `errors.ts` (D1 constraint classification) and `batch.ts`. - `schemas/` — zod/OpenAPI contract split by domain. There is deliberately **no barrel**: import from `schemas/` directly so a module's dependencies are visible. This is why `extensions/v2` has no `interfaces.ts`. - `github/` — outbound GitHub calls, kept out of the persistence modules. +- `email/` (extensions v2) — provider abstraction and templates. Service-root helpers (`revalidate.ts`, `resource-limits.ts`) stay at the service root. `previews/v1` follows the same split (`routes/`, `github/`, `schemas/`, plus `budget.ts` / `cache.ts`). Route modules import `getExtensionsDb`/`getAuth`/`getPlatform` and middleware directly. There is no dependency-injection container; tests drive the real app through `app.request`. @@ -29,8 +30,11 @@ Each service documents its own contract and operational detail in its own `READM - `npm install`: install dependencies. - `npm run dev`: start the local Cloudflare Workers dev server. -- `npm run deploy`: deploy the worker via Wrangler. +- `npm run cf-deploy`: apply remote D1 migrations, then deploy via Wrangler. +- `npm run cf-typegen`: regenerate `worker-configuration.d.ts`. +- `npm run db:migrate:all:local` / `db:migrate:all:remote`: apply all D1 migrations locally / remotely. - `npm run test`: run the default Vitest suite (Workers pool). +- `npm run test:all`: run Workers pool plus Node adapter tests. - `npm run test:node`: run Node adapter tests via `vitest.node.config.ts`. - `npm run test:coverage`: run tests with coverage. - `npm run typecheck`: TypeScript typecheck without emit. @@ -58,13 +62,13 @@ Each service documents its own contract and operational detail in its own `READM ## Configuration & Secrets - Local secrets go in `.dev.vars` (for example `GITHUB_TOKEN="..."`). -- Bindings for D1/KV are defined in `wrangler.jsonc`; keep names aligned with `CloudflareBindings`. +- Bindings, cron, and vars are defined in `wrangler.jsonc` (D1, KV, R2, Durable Objects, rate limiters, service bindings); keep names aligned with `CloudflareBindings`. - Use Wrangler secrets for production tokens instead of committing them. - `ASSERTION_SIGNING_SECRET`: HMAC key the extensions v2 API uses to verify short-lived bearer assertions minted by the extensions site (`src/lib/auth/bearer-assertion.ts`). Not sent over the wire — only signs/verifies server-side in each Worker. Add `ASSERTION_SIGNING_SECRET="..."` to `.dev.vars` for local dev; set via `wrangler secret put ASSERTION_SIGNING_SECRET` in production, matching the value configured in the extensions site's Worker. - `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached catalogue pages after content mutations. Must match the value configured in the extensions site's Worker. Same local/prod setup as `ASSERTION_SIGNING_SECRET`. ## Cache Revalidation -Any endpoint that mutates catalogue-visible content (revision approve/reject, delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, profile deletion via `DELETE /developers/me`, claim approve/reject, extension withdraw) must call `revalidateCatalogue(c)` from `src/services/extensions/v2/revalidate.ts` after a successful write. +Any endpoint that mutates catalogue-visible content (revision approve/reject, moderator-correct, delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, profile deletion via `DELETE /developers/me`, claim approve/reject, extension withdraw) must call `revalidateCatalogue(c)` from `src/services/extensions/v2/revalidate.ts` after a successful write. -Skipping it does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows bound staleness — but changes then take minutes instead of seconds to appear. Profile deletion can remove an approved, extension-less profile, which is still public content; the purge is a no-op today while no cached route carries the `developers` tag, but keeps the endpoint correct if that ever changes. +Skipping it does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows bound staleness — but changes then take minutes instead of seconds to appear. Profile deletion keeps its purge even though the `developers` tag is currently a no-op. From e8856be525dce3dd90e320f7e1f1d399dc9e35c9 Mon Sep 17 00:00:00 2001 From: Adam Daley Date: Fri, 9 Oct 2026 03:50:17 +0100 Subject: [PATCH 4/6] Update extensions v2 API documentation Document the deprecated `status` alias for developer listings and update README links to the current extensions v2 sections. --- AGENTS.md | 2 +- README.md | 6 +++--- src/services/extensions/v2/README.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 2e36dd3..18791ec 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,7 @@ ### Service layout -Small services are flat: `index.ts` + `interfaces.ts` plus narrow helpers (`database.ts`, `db/`, `r2.ts`, `responses.ts`, `dashboard.ts`, `list-cache.ts`, depending on the service). +Small services are flat: `index.ts` + `interfaces.ts` plus narrow helpers (for example `database.ts`, `db/`, `r2.ts`). `extensions/v2` is the reference layout for anything larger, and new services should grow into it rather than inventing a third shape: diff --git a/README.md b/README.md index b4c4ea2..3b28d45 100644 --- a/README.md +++ b/README.md @@ -70,9 +70,9 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d - **KV Namespace** (`AUTH_KV`): Stores the `UPDATE_TOKEN` value for `/versions/v1/update`. - **R2 Bucket** (`DOWNLOAD_BUCKET`): Shared by versions, previews, and stats for release/preview zips — see [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md) and the comment in `wrangler.jsonc` for which bucket this points at and why. - **Durable Object** (`PREVIEW_GITHUB_BUDGET`): Token bucket limiting previews GitHub calls. Policy lives in `src/services/previews/v1/budget.ts`. -- **Rate limiters** (`EXTENSION_WRITE_RATE_LIMITER`, `PROFILE_CREATION_RATE_LIMITER`): Edge pacing for extension writes and profile creation. Durable quotas are enforced in D1; see [extensions v2 README](src/services/extensions/v2/README.md#extension-resource-admission-and-retention). +- **Rate limiters** (`EXTENSION_WRITE_RATE_LIMITER`, `PROFILE_CREATION_RATE_LIMITER`): Edge pacing for extension writes and profile creation. Durable quotas are enforced in D1; see [extensions v2 README](src/services/extensions/v2/README.md#limits-and-budgets). - **Service binding** (`EXTENSIONS_FRONTEND`): Private binding to the Extensions site worker, used to purge cached catalogue pages after mutations. -- **Cron** (`0 * * * *`): Hourly extension resource maintenance. See [extensions v2 README](src/services/extensions/v2/README.md#maintenance-and-monitoring). +- **Cron** (`0 * * * *`): Hourly extension resource maintenance. See [extensions v2 README](src/services/extensions/v2/README.md#revisions-and-retention). ### Environment Variables @@ -85,7 +85,7 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d - `EXTENSIONS_V2_EMAIL_REPLY_TO`: Optional Reply-To address for a monitored inbox. Omit when unset (no Reply-To header is sent) — notification emails do not invite replies. - `EXTENSIONS_V2_MXROUTE_SERVER`, `EXTENSIONS_V2_MXROUTE_USERNAME`, `EXTENSIONS_V2_MXROUTE_PASSWORD`: Mailbox credentials for the MXroute SMTP API (`https://smtpapi.mxroute.com/`). Required when `EXTENSIONS_V2_EMAIL_PROVIDER=mxroute`. - `EXTENSIONS_V2_RESEND_API_KEY`: API key for Resend. Required when `EXTENSIONS_V2_EMAIL_PROVIDER=resend`. -- `EXTENSIONS_RETENTION_MODE`: `dry-run` (report only) or `compact` (prune old revision bodies). See [extensions v2 README](src/services/extensions/v2/README.md#maintenance-and-monitoring). +- `EXTENSIONS_RETENTION_MODE`: `dry-run` (report only) or `compact` (prune old revision bodies). See [extensions v2 README](src/services/extensions/v2/README.md#revisions-and-retention). Only extensions v2 consumes these. For the assertion format and the rotation procedure, see [its README](src/services/extensions/v2/README.md#authentication). diff --git a/src/services/extensions/v2/README.md b/src/services/extensions/v2/README.md index dc7a7a1..81b8813 100644 --- a/src/services/extensions/v2/README.md +++ b/src/services/extensions/v2/README.md @@ -49,7 +49,7 @@ Owner/moderator extension reads return four independent fields, not a derived st `delisted` is orthogonal to all three: it hides the row from public reads without touching them. Adopted rows have no revisions; a live extension with no review history is normal. -`GET /extensions/{id}/revisions` lists one extension's history (owner/moderator, newest first). `GET /revisions` is the global review queue (moderator only, `?status=` defaults to `pending`, oldest first). `GET /developers?scope=` (`all` default, `unapproved` for the queue) and `GET /developers/claims?scope=` (`mine`, `pending`) follow the same pattern. `PATCH /users/me` returns the full account projection, like `GET /users/me`. +`GET /extensions/{id}/revisions` lists one extension's history (owner/moderator, newest first). `GET /revisions` is the global review queue (moderator only, `?status=` defaults to `pending`, oldest first). `GET /developers?scope=` (`all` default, `unapproved` for the queue; `?status=` stays as a deprecated alias and 422s when it disagrees with `?scope=`) and `GET /developers/claims?scope=` (`mine`, `pending`) follow the same pattern. `PATCH /users/me` returns the full account projection, like `GET /users/me`. Anonymous reads are cacheable (`Cache-Control: public`); authenticated reads send `Vary: Authorization`. From f50e3cec3798ee5680b2527501559325f6444ef4 Mon Sep 17 00:00:00 2001 From: Adam Daley Date: Fri, 9 Oct 2026 03:55:29 +0100 Subject: [PATCH 5/6] Address review feedback on docs --- README.md | 2 +- src/services/extensions/v2/README.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 3b28d45..c86d163 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ The worker exposes five services: See [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md). - **Stats** (`/stats/v1`) - Aggregates release data from the versions service into charts and JSON. No separate GitHub calls. + Aggregates release data through the versions service's shared releases cache into charts and JSON; a cold cache can trigger a GitHub fetch through that shared path. See [`src/services/stats/v1/README.md`](src/services/stats/v1/README.md). ## Architecture diff --git a/src/services/extensions/v2/README.md b/src/services/extensions/v2/README.md index 81b8813..bd7070f 100644 --- a/src/services/extensions/v2/README.md +++ b/src/services/extensions/v2/README.md @@ -28,7 +28,7 @@ An edit cannot rename an extension or move it to another developer. A user owns ## Reads -`GET /extensions/{id}` and `GET /developers/{id}` are role-aware: anonymous callers get the published projection (404 hides drafts and delisted rows); owners and moderators get the full object. +`GET /extensions/{id}` and `GET /developers/{id}` are role-aware: anonymous callers get the published projection; owners and moderators get the full object. For extensions, 404 hides drafts and delisted rows. `GET /extensions?scope=` selects the projection: @@ -51,7 +51,7 @@ Owner/moderator extension reads return four independent fields, not a derived st `GET /extensions/{id}/revisions` lists one extension's history (owner/moderator, newest first). `GET /revisions` is the global review queue (moderator only, `?status=` defaults to `pending`, oldest first). `GET /developers?scope=` (`all` default, `unapproved` for the queue; `?status=` stays as a deprecated alias and 422s when it disagrees with `?scope=`) and `GET /developers/claims?scope=` (`mine`, `pending`) follow the same pattern. `PATCH /users/me` returns the full account projection, like `GET /users/me`. -Anonymous reads are cacheable (`Cache-Control: public`); authenticated reads send `Vary: Authorization`. +Anonymous detail reads are cacheable (`Cache-Control: public`); detail reads that vary by caller send `Vary: Authorization`. ## Pagination From b07fd3e45f8b5bbc212e431cdc92ff03388cbc95 Mon Sep 17 00:00:00 2001 From: Adam Daley Date: Fri, 9 Oct 2026 03:55:54 +0100 Subject: [PATCH 6/6] Address Codex review feedback on extensions v2 docs --- src/services/extensions/v2/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/services/extensions/v2/README.md b/src/services/extensions/v2/README.md index bd7070f..153ba21 100644 --- a/src/services/extensions/v2/README.md +++ b/src/services/extensions/v2/README.md @@ -86,7 +86,7 @@ Recipients: developer `contact_email`, falling back to the owning account's emai ## Limits and budgets -Request size is counted on the raw stream before JSON parsing. Raw limit is **512 KiB** (`413 BODY_TOO_LARGE`); normalized JSON content limit is **256 KiB**. New slug ids are at most 200 characters. +Content writes (`POST /extensions`, `PUT /extensions/{id}`, moderator-correct) count the raw request stream before JSON parsing. Raw limit is **512 KiB** (`413 BODY_TOO_LARGE`); normalized JSON content limit is **256 KiB**. New slug ids are at most 200 characters. `EXTENSION_WRITE_RATE_LIMITER` paces IP and account attempts at 60/minute, including validation failures. It is approximate edge pacing, not the durable quota. Missing pacing fails closed (`503 ADMISSION_UNAVAILABLE`). @@ -121,11 +121,12 @@ Uses D1 binding `DB_EXTENSIONS`, shared with v1 (read-only there). This service - `0020`: guard check; fails if an adopted developer id shadows a static route (`me`/`claims`/`unapproved`). - `0021`: rebuilds `extensions`, replaces `extension_submissions` with `extension_revisions`. Ordering is load-bearing (never drop a table with children); covered by `migrations.test.ts`. Refuses unmigratable data with named checks: -| Failure | Meaning | -| -------------------------------------------- | ------------------------------------------------ | -| `extension_ids_must_not_differ_only_by_case` | Two ids collide under `idx_extensions_id_nocase` | -| `submission_target_ids_must_not_be_reserved` | A submission targets a reserved static-route id | -| `extension_references_must_resolve` | Rebuild would carry a dangling reference through | +| Failure | Meaning | +| --------------------------------------------- | ------------------------------------------------------- | +| `extension_ids_must_not_differ_only_by_case` | Two ids collide under `idx_extensions_id_nocase` | +| `extension_references_must_resolve` | An extension names a developer id with no developer row | +| `submissions_must_name_an_existing_developer` | A submission names a developer id with no developer row | +| `submission_target_ids_must_not_be_reserved` | A submission targets a reserved static-route id | Reconcile and re-run; the migration touches nothing before these checks.