Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 19 additions & 45 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,41 +10,31 @@

### Service layout

Small services are flat: `index.ts`, `interfaces.ts`, optionally `database.ts`
and `db/` (see `central-alerts/v1`, `versions/v1`, `stats/v1`).
Small services are flat: `index.ts` + `interfaces.ts` plus narrow helpers (for example `database.ts`, `db/`, `r2.ts`).

`extensions/v2` is the reference layout for anything larger, and new services
should grow into it rather than inventing a third shape:
`extensions/v2` is the reference layout for anything larger, and new services should grow into it rather than inventing a third shape:

- `index.ts` — app assembly only: middleware, route registration, OpenAPI
document. Route registration order is load-bearing where static paths must
beat parameter paths; those cases carry comments.
- `index.ts` — app assembly only: middleware, route registration, OpenAPI document. Route registration order is load-bearing where static paths must beat parameter paths; those cases carry comments.
- `middleware.ts` — service-specific Hono middleware.
- `routes/` — one module per route group, each exporting `register*Routes(app)`.
`routes/errors.ts` maps domain error codes to HTTP status; `routes/app.ts`
holds the typed app alias.
- `db/` — `schema.ts`, `migrations/`, one `*Database` class per workflow, plus
`errors.ts` (D1 constraint classification) and `batch.ts`.
- `schemas/` — zod/OpenAPI contract split by domain. There is deliberately **no
barrel**: import from `schemas/<domain>` directly so a module's dependencies
are visible. This is why `extensions/v2` has no `interfaces.ts`.
- `routes/` — one module per route group, each exporting `register*Routes(app)`. `routes/errors.ts` maps domain error codes to HTTP status; `routes/app.ts` holds the typed app alias.
- `db/` — `schema.ts`, `migrations/`, one `*Database` class per workflow, plus `errors.ts` (D1 constraint classification) and `batch.ts`.
- `schemas/` — zod/OpenAPI contract split by domain. There is deliberately **no barrel**: import from `schemas/<domain>` directly so a module's dependencies are visible. This is why `extensions/v2` has no `interfaces.ts`.
- `github/` — outbound GitHub calls, kept out of the persistence modules.
- `email/` (extensions v2) — provider abstraction and templates. Service-root helpers (`revalidate.ts`, `resource-limits.ts`) stay at the service root. `previews/v1` follows the same split (`routes/`, `github/`, `schemas/`, plus `budget.ts` / `cache.ts`).

Route modules import `getExtensionsDb`/`getAuth`/`getPlatform` and middleware
directly. There is no dependency-injection container; tests drive the real app
through `app.request`.
Route modules import `getExtensionsDb`/`getAuth`/`getPlatform` and middleware directly. There is no dependency-injection container; tests drive the real app through `app.request`.

Each service documents its own contract and operational detail in its own
`README.md` (`src/services/<name>/<version>/README.md`). Keep API behaviour
there rather than here or in the root README: this file is for conventions that
apply when modifying the code.
Each service documents its own contract and operational detail in its own `README.md` (`src/services/<name>/<version>/README.md`). Keep API behaviour there rather than here or in the root README: this file is for conventions that apply when modifying the code.

## Build, Test, and Development Commands

- `npm install`: install dependencies.
- `npm run dev`: start the local Cloudflare Workers dev server.
- `npm run deploy`: deploy the worker via Wrangler.
- `npm run cf-deploy`: apply remote D1 migrations, then deploy via Wrangler.
- `npm run cf-typegen`: regenerate `worker-configuration.d.ts`.
- `npm run db:migrate:all:local` / `db:migrate:all:remote`: apply all D1 migrations locally / remotely.
- `npm run test`: run the default Vitest suite (Workers pool).
- `npm run test:all`: run Workers pool plus Node adapter tests.
- `npm run test:node`: run Node adapter tests via `vitest.node.config.ts`.
- `npm run test:coverage`: run tests with coverage.
- `npm run typecheck`: TypeScript typecheck without emit.
Expand Down Expand Up @@ -72,29 +62,13 @@ apply when modifying the code.
## Configuration & Secrets

- Local secrets go in `.dev.vars` (for example `GITHUB_TOKEN="..."`).
- Bindings for D1/KV are defined in `wrangler.jsonc`; keep names aligned with `CloudflareBindings`.
- Bindings, cron, and vars are defined in `wrangler.jsonc` (D1, KV, R2, Durable Objects, rate limiters, service bindings); keep names aligned with `CloudflareBindings`.
- Use Wrangler secrets for production tokens instead of committing them.
- `ASSERTION_SIGNING_SECRET`: HMAC key the extensions v2 API uses to verify short-lived
bearer assertions minted by the extensions site (`src/lib/auth/bearer-assertion.ts`).
Not sent over the wire — only signs/verifies server-side in each Worker. Add
`ASSERTION_SIGNING_SECRET="..."` to `.dev.vars` for local dev; set via
`wrangler secret put ASSERTION_SIGNING_SECRET` in production, matching the value
configured in the extensions site's Worker.
- `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the
extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service
binding) to purge the site's CDN-cached catalogue pages after content mutations.
Must match the value configured in the extensions site's Worker. Same local/prod
setup as `ASSERTION_SIGNING_SECRET`.
- `ASSERTION_SIGNING_SECRET`: HMAC key the extensions v2 API uses to verify short-lived bearer assertions minted by the extensions site (`src/lib/auth/bearer-assertion.ts`). Not sent over the wire — only signs/verifies server-side in each Worker. Add `ASSERTION_SIGNING_SECRET="..."` to `.dev.vars` for local dev; set via `wrangler secret put ASSERTION_SIGNING_SECRET` in production, matching the value configured in the extensions site's Worker.
- `EXTENSIONS_REVALIDATE_SECRET`: bearer token the extensions v2 API sends to the extensions site's `POST /api/revalidate` (over the `EXTENSIONS_FRONTEND` service binding) to purge the site's CDN-cached catalogue pages after content mutations. Must match the value configured in the extensions site's Worker. Same local/prod setup as `ASSERTION_SIGNING_SECRET`.

## Cache Revalidation

Any endpoint that mutates catalogue-visible content (revision approve/reject,
delist/relist, developer approve, developer profile upsert via `PUT /developers/me`,
profile deletion via `DELETE /developers/me`, claim approve/reject, extension
withdraw) must call `revalidateCatalogue(c)` from
`src/services/extensions/v2/revalidate.ts` after a successful write. Skipping it
does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows
bound staleness — but changes then take minutes instead of seconds to appear.
Profile deletion can remove an approved, extension-less profile, which is still
public content; the purge is a no-op today while no cached route carries the
`developers` tag, but keeps the endpoint correct if that ever changes.
Any endpoint that mutates catalogue-visible content (revision approve/reject, moderator-correct, delist/relist, developer approve, developer profile upsert via `PUT /developers/me`, profile deletion via `DELETE /developers/me`, claim approve/reject, extension withdraw) must call `revalidateCatalogue(c)` from `src/services/extensions/v2/revalidate.ts` after a successful write.

Skipping it does not break correctness — the site's `maxAge`/`stale-while-revalidate` windows bound staleness — but changes then take minutes instead of seconds to appear. Profile deletion keeps its purge even though the `developers` tag is currently a no-op.
42 changes: 33 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,32 @@
# FOSSBilling API Worker

This is the API service that acts as the central hub for FOSSBilling instances. It handles version checks, update information, and broadcasts system-wide alerts.
This is the API service that acts as the central hub for FOSSBilling instances. It serves version checks, system-wide alerts, the extensions catalogue, preview builds, and release stats.

Everything is built on [Hono](https://hono.dev), making it lightweight and fast. While we currently deploy this to Cloudflare Workers, the code is designed to be platform-agnostic.

## What it does

The worker exposes four main services:
The worker exposes five services:

- **Versions Service** (`/versions/v1`)
- **Versions** (`/versions/v1`)
The source of truth for FOSSBilling updates. It fetches release data from GitHub, caches it for performance, and helps instances decide if they need to update.

- **Central Alerts** (`/central-alerts/v1`)
Allows the project to push critical notifications to all FOSSBilling installations—useful for security hotfixes or major announcements.

- **Extensions** (`/extensions/v1`, `/extensions/v2`)
Owns the complete Extensions domain and its `DB_EXTENSIONS` schema. The separate Extensions site keeps OIDC/session state but accesses this domain through the
Owns the complete Extensions domain and its `DB_EXTENSIONS` schema. `v2` is authoritative; `v1` is legacy read-only. The separate Extensions site keeps OIDC/session state but accesses this domain through the
generated HTTPS API client; it must not bind or migrate `DB_EXTENSIONS`.
See [`src/services/extensions/v2/README.md`](src/services/extensions/v2/README.md).

- **Previews** (`/previews/v1`)
Resolves FOSSBilling preview builds — the current main preview and per-PR/per-commit builds produced by FOSSBilling/FOSSBilling's GitHub Actions workflows. Read-only; GitHub Actions and R2 are the sources of truth, not this service.
See [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md).

- **Stats** (`/stats/v1`)
Aggregates release data through the versions service's shared releases cache into charts and JSON; a cold cache can trigger a GitHub fetch through that shared path.
See [`src/services/stats/v1/README.md`](src/services/stats/v1/README.md).

## Architecture

We've structured the app to separate the core logic from the specific runtime environment (Cloudflare, Node, etc.).
Expand All @@ -35,6 +39,8 @@ We've structured the app to separate the core logic from the specific runtime en
- `src/lib/adapters/cloudflare`: Real implementations using KV and D1.
- `src/lib/adapters/node`: Reference implementations (useful for testing or alternative deployments).

Cron and Durable Object code lives with its owning service and is wired in `src/app/index.ts`.

## APIs

Each service documents its own endpoints and behaviour:
Expand Down Expand Up @@ -62,7 +68,11 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d
Migrations are owned by extensions v2 and applied only from this repository — see [its README](src/services/extensions/v2/README.md#database) for the migration and adoption procedure.
- **KV Namespace** (`CACHE_KV`): Caches GitHub API responses so we don't hit rate limits.
- **KV Namespace** (`AUTH_KV`): Stores the `UPDATE_TOKEN` value for `/versions/v1/update`.
- **R2 Bucket** (`DOWNLOAD_BUCKET`): Backs `/previews/v1/main` — see [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md) and the comment in `wrangler.jsonc` for which bucket this points at and why.
- **R2 Bucket** (`DOWNLOAD_BUCKET`): Shared by versions, previews, and stats for release/preview zips — see [`src/services/previews/v1/README.md`](src/services/previews/v1/README.md) and the comment in `wrangler.jsonc` for which bucket this points at and why.
- **Durable Object** (`PREVIEW_GITHUB_BUDGET`): Token bucket limiting previews GitHub calls. Policy lives in `src/services/previews/v1/budget.ts`.
- **Rate limiters** (`EXTENSION_WRITE_RATE_LIMITER`, `PROFILE_CREATION_RATE_LIMITER`): Edge pacing for extension writes and profile creation. Durable quotas are enforced in D1; see [extensions v2 README](src/services/extensions/v2/README.md#limits-and-budgets).
- **Service binding** (`EXTENSIONS_FRONTEND`): Private binding to the Extensions site worker, used to purge cached catalogue pages after mutations.
- **Cron** (`0 * * * *`): Hourly extension resource maintenance. See [extensions v2 README](src/services/extensions/v2/README.md#revisions-and-retention).

### Environment Variables

Expand All @@ -75,6 +85,7 @@ We use [Cloudflare D1](https://developers.cloudflare.com/d1/) and [KV](https://d
- `EXTENSIONS_V2_EMAIL_REPLY_TO`: Optional Reply-To address for a monitored inbox. Omit when unset (no Reply-To header is sent) — notification emails do not invite replies.
- `EXTENSIONS_V2_MXROUTE_SERVER`, `EXTENSIONS_V2_MXROUTE_USERNAME`, `EXTENSIONS_V2_MXROUTE_PASSWORD`: Mailbox credentials for the MXroute SMTP API (`https://smtpapi.mxroute.com/`). Required when `EXTENSIONS_V2_EMAIL_PROVIDER=mxroute`.
- `EXTENSIONS_V2_RESEND_API_KEY`: API key for Resend. Required when `EXTENSIONS_V2_EMAIL_PROVIDER=resend`.
- `EXTENSIONS_RETENTION_MODE`: `dry-run` (report only) or `compact` (prune old revision bodies). See [extensions v2 README](src/services/extensions/v2/README.md#revisions-and-retention).

Only extensions v2 consumes these. For the assertion format and the rotation procedure, see [its README](src/services/extensions/v2/README.md#authentication).

Expand All @@ -101,8 +112,7 @@ npm install
2. Apply migrations to the local D1 databases:

```bash
npm run db:migrate:extensions-v2:local
npm run db:migrate:central-alerts:local
npm run db:migrate:all:local
```

3. (Optional) Store an update token in KV for `/versions/v1/update`:
Expand All @@ -121,8 +131,22 @@ You can now hit endpoints at `http://localhost:8787`.

### Testing

We use Vitest for testing. The suite includes unit tests for the endpoints and integration tests using the platform adapters.
We use Vitest. Unit tests run under the Workers pool; Node adapter tests use a separate config.

```bash
npm run test # workers pool
npm run test:all # workers + node adapters
```

```bash
npm run typecheck
npm run lint
```

### Deploy

```bash
npm run test
npm run cf-deploy
```

This applies remote D1 migrations, then deploys with Wrangler.
15 changes: 4 additions & 11 deletions src/services/central-alerts/v1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,11 @@ The Central Alerts service provides targeted notifications to FOSSBilling instal

### GET `/list`

Retrieve all alerts in the system. Optional `limit` (integer 1–100) and
`offset` enable pagination. An unusable limit keeps the full-list response;
`offset` without a usable limit returns 422. An omitted or unusable offset
with a usable limit defaults to zero.
Retrieve all alerts in the system. Optional `limit` (integer 1–100) and `offset` enable pagination. An unusable limit keeps the full-list response; `offset` without a usable limit returns 422. An omitted or unusable offset with a usable limit defaults to zero.

Successful responses are edge-cached for 60 seconds by the effective page.
Unknown query parameters and equivalent pagination spellings reuse the same
entry. This endpoint is public: Authorization does not affect its response or
bypass its cache. Validation failures and database errors are not cached.
Successful responses are edge-cached for 60 seconds by the effective page. Unknown query parameters and equivalent pagination spellings reuse the same entry. This endpoint is public: Authorization does not affect its response or bypass its cache. Validation failures and database errors are not cached.

When `limit` is usable the response adds a `pagination` object next to
`alerts`:
When `limit` is usable the response adds a `pagination` object next to `alerts`:

```json
{
Expand Down Expand Up @@ -79,4 +72,4 @@ The `type` field accepts: `success`, `info`, `warning`, `danger`

## Database

Uses D1 database binding `DB_CENTRAL_ALERTS`. Initialize with the setup script in `src/services/central-alerts/v1/scripts/`.
Uses D1 database binding `DB_CENTRAL_ALERTS`. Apply migrations from `db/migrations` with `npm run db:migrate:central-alerts:local` / `:remote`.
Loading
Loading