Self-hostable package registry for the Sema programming language. Ships as a single binary with SQLite, serves both a web UI and a REST API for CLI clients.
jake dev # start locally (cargo) on a fresh DB and seed it
jake docker # build + start in Docker on a fresh DB and seed itBoth start the registry on http://localhost:3000 and load the
demo data (see Local Development below). If port 3000 is busy they
bump to the next free port (and point the seed at the same one) — the chosen URL is
printed at startup. Override with jake dev port=4000. jake dev runs the server in the
foreground (Ctrl-C to stop); jake docker runs it detached and tails the logs
(jake down to stop the container).
To run the server without seeding:
jake run # locally, no reset/seed
docker compose up --buildRun jake -l to list all targets.
jake dev / jake docker seed a fresh database, then start the server on it. The
seeder (src/bin/seed.rs) writes straight to DATABASE_URL through the app's own
entities, so it's engine-portable (SQLite/Postgres/MySQL), fast, and needs no
running server. Seeded users have real password hashes (they can log in), featured
packages have real downloadable blobs, and it prints a working API token.
The reproducible demo dataset:
- Users:
helge(admin),kari,magnus,ingrid,olav,spambot(banned), plus realistic bulk accounts. Every seeded user has the password123123123. Admin login:helge/123123123, panel at/admin. - Packages: featured
sema-http,sema-json,sema-router,sema-sql,sema-cli,sema-async,sema-test,sema-crypto,sema-csv,sema-log(multiple versions, some yanked, READMEs, real blobs) plus dozens of realistic packages. - Downloads, reports, audit log: populated with a believable distribution.
jake seed # small realistic dev set (wipes + reseeds)
jake seed-stress # bulk: ~1k users, ~500 packages
# Any engine — just point DATABASE_URL at it:
DATABASE_URL="postgres://sema:sema@localhost:5432/sema" jake seed
# Or run the binary directly for flags (--fresh, --large):
cargo run --features seed --bin seed -- --fresh --largeFor scale testing, --huge seeds 20k users and 1M packages (~4.5M versions) in
under a minute on SQLite; SEED_USERS / SEED_PACKAGES override any preset.
Uploaded releases get an API reference at /packages/{name}/{version}/api.
The package page links to it. The reference includes a searchable export index,
signatures with distinct argument styling, syntax-highlighted definitions, and
links to the stored source with line numbers. Two other formats use the same
stored document:
- Append
.mdto the reference URL for Markdown (text/markdown). - Append
/printfor a standalone HTML page with inline styles and no external assets. Save that page for offline reading or print it from the browser.
Extraction uses the pinned Sema reader, without running package code. It reads
the explicit [package].entrypoint in sema.toml, literal top-level modules and
their export lists, direct function declarations, and leading string docstrings
with a following function body. It does not invent parameter types, examples,
or a documentation syntax. Aliases, computed definitions, and ambiguous bindings
are shown as unresolved. Unsupported source produces an explanation and a README
link; it does not prevent publication.
Each document records the archive checksum, source checksum, and parser version. The parser accepts at most 16 MiB of expanded archive data, 4,096 archive entries, 512 KiB per source file, and 512 exports. Documents are stored once per release. GitHub-linked releases without a registry-stored archive cannot be documented this way; the registry does not fetch mutable repository files to fill gaps.
Backfill existing stored uploads after deploying the migration:
sema-pkg package backfill-api-docsThe command verifies each stored archive's SHA-256, skips existing documents and
GitHub-linked releases, and processes versions in bounded batches. Set
API_DOCS_ENABLED=false to disable generation, links, routes, and backfill.
For a local preview, start the registry with a separate database and blob directory on port 4177, then run:
node scripts/preview-api-docs.mjs http://127.0.0.1:4177The script downloads checksum-verified public releases of sema-retry,
sema-cache, and sema-medley, and publishes them to the local server through
its normal API. It accepts only loopback targets and prints the reference URLs.
Traces (OpenTelemetry) and metrics (Prometheus) are built in and off until configured — nothing is exported by default. See DEPLOYMENT.md for a full local Jaeger + Prometheus
- Grafana stack.
Traces — pick an exporter with OTEL_TRACES_EXPORTER:
jake trace # → traces.jsonl (one span per line)
OTEL_TRACES_EXPORTER=otlp \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 sema-pkg # → Jaeger/TempoRequests, handlers, and DAL queries nest into a request → handler → query tree;
each span carries duration_ms and the source code.file.path/line. For the
file exporter, render the slowest spans with jq:
jq -r '[.name,(.duration_ms|tostring)+"ms",(.attributes.op//"")]|@tsv' traces.jsonl \
| sort -t$'\t' -k2 -rn | column -t -s$'\t'Metrics — METRICS_ENABLED=true serves Prometheus RED metrics at /metrics
(http_requests_total, http_request_duration_seconds, http_requests_in_flight),
labelled by matched route.
RUST_LOG sets log verbosity; OTEL_LOG sets span capture independently, so
quiet logs and rich traces coexist.
All configuration is via environment variables with sensible defaults:
| Variable | Default | Description |
|---|---|---|
HOST |
0.0.0.0 |
Bind address |
PORT |
3000 |
Listen port |
DATABASE_URL |
sqlite://data/registry.db?mode=rwc |
Database URL. The engine is inferred from the scheme — sqlite:, postgres:, or mysql: (see Database engines). |
DATABASE_MAX_CONNECTIONS |
10 |
Connection pool size (read concurrency). |
BLOB_DIR |
data/blobs |
Directory for package tarballs (used when S3 is not configured) |
BLOB_S3_BUCKET |
— | Bucket for S3-compatible blob storage (Cloudflare R2, MinIO, S3). When set, tarballs go to object storage instead of BLOB_DIR — required for stateless / multi-instance deploys. |
BLOB_S3_ENDPOINT |
— | S3 endpoint URL (e.g. https://<account>.r2.cloudflarestorage.com). Omit for AWS S3. |
BLOB_S3_REGION |
auto |
Region. auto for R2. |
BLOB_S3_ACCESS_KEY_ID |
— | Access key (required when BLOB_S3_BUCKET is set) |
BLOB_S3_SECRET_ACCESS_KEY |
— | Secret key (required when BLOB_S3_BUCKET is set) |
BASE_URL |
http://localhost:3000 |
Public URL (used in links; enables Secure session cookies when https://) |
MAX_TARBALL_BYTES |
52428800 (50 MB) |
Max upload size |
MAX_DEPENDENCIES |
64 |
Max dependencies per published version |
API_DOCS_ENABLED |
true |
Generate and serve API references for stored releases; also controls backfill. |
GITHUB_CLIENT_ID |
— | GitHub OAuth app client ID (optional) |
GITHUB_CLIENT_SECRET |
— | GitHub OAuth app secret (optional) |
OAUTH_TOKEN_KEY |
— | 32-byte key encrypting stored GitHub tokens. Required when GitHub OAuth is enabled — the server refuses to boot if left at the insecure default. |
RATE_LIMIT_ENABLED |
true |
IP-keyed request rate limiting (see DEPLOYMENT.md). Disable only behind a gateway that limits for you. |
RATE_LIMIT_RPS |
20 |
Sustained requests/sec per IP on the general/write API |
RATE_LIMIT_BURST |
40 |
Burst allowance per IP before throttling |
RATE_LIMIT_READ_RPS |
100 |
Sustained requests/sec per IP on the install path (metadata + download); generous so multi-package installs aren't throttled |
RATE_LIMIT_READ_BURST |
500 |
Burst allowance per IP on the install path |
The registry runs on SQLite, PostgreSQL, or MySQL from the same
binary — the engine is chosen by the DATABASE_URL scheme:
DATABASE_URL="sqlite://data/registry.db?mode=rwc" # default
DATABASE_URL="postgres://user:pass@host:5432/sema" # PostgreSQL
DATABASE_URL="mysql://user:pass@host:3306/sema" # MySQLOn startup the server infers the backend, applies SQLite-only tuning (WAL) where
relevant, and runs the schema migrations (src/migration/, SeaORM programmatic
migrations that emit correct DDL per engine). No manual schema step is needed.
After deploying the release-documentation migration, run this once against the production database and its original upload blob store to capture verified historical release READMEs and changelogs:
sema-pkg package backfill-readmesIt processes archive metadata in bounded batches, checks each archive's stored SHA-256, and skips GitHub-linked releases because their original archives are not stored by the registry.
All database access goes through the Data Access Layer in src/dal/ (one module
per aggregate: packages, versions, owners, deps, users, sessions,
tokens, reports, audit_log, oauth, downloads, plus admin read models
and a time helper). Handlers never touch SQL directly, and the DAL avoids
engine-specific constructs — timestamps are generated in Rust, upserts use
SeaORM's on_conflict, and any raw SQL is standard and parameterized. This is
what keeps all three engines behaving identically; add new queries to the DAL,
not to handlers.
Run the suite against all three engines (Docker):
jake all-drivers # SQLite + PostgreSQL + MySQL via docker-compose.test.yml| Method | Path | Auth | Description |
|---|---|---|---|
POST |
/api/v1/auth/register |
— | Create account {username, email, password} |
POST |
/api/v1/auth/login |
— | Sign in {username, password} |
POST |
/api/v1/auth/logout |
— | Clear session |
| Method | Path | Auth | Description |
|---|---|---|---|
POST |
/api/v1/tokens |
Session | Create API token {name} |
GET |
/api/v1/tokens |
Session | List your tokens |
DELETE |
/api/v1/tokens/{id} |
Session | Revoke a token |
| Method | Path | Auth | Description |
|---|---|---|---|
GET |
/api/v1/search?q=&page=&per_page= |
— | Search packages |
GET |
/api/v1/packages/{name} |
— | Package metadata + versions |
PUT |
/api/v1/packages/{name}/{version} |
Bearer | Publish version (multipart: tarball + metadata) |
GET |
/api/v1/packages/{name}/{version}/download |
— | Download tarball |
POST |
/api/v1/packages/{name}/{version}/yank |
Bearer | Yank a version |
| Method | Path | Auth | Description |
|---|---|---|---|
GET |
/api/v1/packages/{name}/owners |
— | List owners |
PUT |
/api/v1/packages/{name}/owners |
Bearer | Add owner {username} |
DELETE |
/api/v1/packages/{name}/owners |
Bearer | Remove owner {username} |
| Method | Path | Auth | Description |
|---|---|---|---|
POST |
/api/v1/packages/link |
Session | Link a GitHub repo {repo_url} |
POST |
/api/v1/packages/{name}/sync |
Session | Manual re-sync from GitHub (owner only) |
POST |
/api/v1/webhooks/github |
HMAC | Webhook receiver for tag events |
Link a GitHub repository to automatically publish packages from semver tags.
- GitHub OAuth configured (
GITHUB_CLIENT_ID,GITHUB_CLIENT_SECRET) OAUTH_TOKEN_KEYset to a random 32-character string (used to encrypt stored GitHub tokens)
- User connects their GitHub account via OAuth
- User pastes a repo URL on the
/linkpage - Registry validates the repo contains a
sema.toml, then imports existing semver tags as versions - A webhook is registered on the repo — new semver tags are published automatically
Git tags are mapped to package versions: v1.0.0 → 1.0.0. Tags that don't match semver (e.g., nightly, latest) are skipped.
Packages may declare a SemVer requirement in sema.toml:
[package]
sema_version_req = ">=1.34.0"The registry validates and normalizes this field for both CLI uploads and GitHub-linked releases. Clients use it to skip incompatible versions during unversioned installs and to reject incompatible explicit or locked installs before replacing local package files.
For a GitHub-linked package the requirement is read from sema.toml at each
tag, so every release records the requirement it was published with and
editing the field does not rewrite past releases. A tag whose sema.toml is
missing or unreadable records no requirement; a tag whose sema.toml parses but
holds an invalid requirement is reported in the package's sync log.
A package is either CLI-uploaded or GitHub-linked, never both. Once a package is linked to a repo, it cannot be published via sema publish, and vice versa.
- Build:
cargo build --release - Copy
target/release/sema-pkg,templates/, andstatic/to your server (schema migrations are compiled into the binary and run automatically on startup) - Set
DATABASE_URL,BLOB_DIR,BASE_URL(use anhttps://URL so session cookies are markedSecure), and — if using GitHub OAuth — a uniqueOAUTH_TOKEN_KEY - Run
sema-pkgbehind a reverse proxy (nginx/caddy) with TLS
Or use the Docker image:
docker compose up -dData is stored in ./data/ (SQLite DB + blob files). Back up this directory.
Operate the registry from the CLI (works on any backend, no manual DB edits):
sema-pkg admin create <user> <email> <pw> # create the first admin (API can't)
sema-pkg admin promote|demote|ban|unban <user>
sema-pkg admin reset-password <user> <pw> # recover a locked-out account
sema-pkg admin token-create <user> [name] # mint an API token (e.g. for a bot account)
sema-pkg admin revoke-tokens <user> # after a token leak
sema-pkg package yank <name> <version> # moderation / takedowns
sema-pkg package remove <name>
sema-pkg stats # user/package/download counts
sema-pkg doctor # check DB, blob store, configFor production hosting — Fly.io (recommended: SQLite + Tigris + built-in Litestream backup), Docker Compose, a bare binary + systemd, or managed Postgres for horizontal scale — see DEPLOYMENT.md.
- Create a GitHub OAuth App at https://github.com/settings/developers
- Set callback URL to
{BASE_URL}/auth/github/callback - Set
GITHUB_CLIENT_IDandGITHUB_CLIENT_SECRETenvironment variables - Restart the server — the GitHub login button will appear automatically