Skip to content

fix(og): stop the public OG endpoint blocking the event loop, and bound it - #434

Open
QSchlegel wants to merge 1 commit into
preprodfrom
claude/daily-2026-11-03-og-endpoint-bounds
Open

QSchlegel wants to merge 1 commit into
preprodfrom
claude/daily-2026-11-03-og-endpoint-bounds

Conversation

@QSchlegel

Copy link
Copy Markdown
Collaborator

What

GET /api/v1/og is public and unauthenticated, and each call fetches up to 1 MiB from an allowlisted host. This PR bounds the endpoint:

  1. Linear-time meta parsing. The old per-field regexes (<meta[^>]+property=…[^>]+content=…) rescanned the rest of the page from every <meta opening. They are replaced by src/lib/html/metaTags.ts, a single-pass tag and attribute tokenizer. It caps each tag at 4 KiB, ignores > inside quoted values, and resumes from where a dropped tag stopped.
  2. GET only. Any other method gets 405 with Allow: GET.
  3. Rate limit. 20 requests per minute per IP, checked before any DNS lookup or fetch.
  4. Gateway errors instead of 500s. A failing or redirecting upstream returns 502 and a timeout returns 504. The raw internal error text (Upstream 404, connect ECONNREFUSED …) is no longer echoed to the caller; unexpected errors are only logged.

Why

The regexes are quadratic in the number of unclosed <meta openings:

Input Before After
128 KiB of unclosed <meta openings, one regex ~350 ms —
1 MiB of the same, whole handler (six lookups) > 2 min (the jest run of the old handler had to be killed) ~16 ms
1 MiB, attribute names near the 4 KiB cap with no = — ~14 ms
1 MiB of <meta a="x> (unclosed quotes) — ~20 ms
1 MiB of valid tags — ~21 ms

All of that time is spent on the event loop of a public endpoint. With the default allowlist an attacker needs to control the HTML on an allowlisted host. With the supported OG_ALLOWED_HOSTS=* setting, any page they host is enough.

Behaviour changes callers will notice

  • 405, 429, 502 and 504 replace the former blanket 500. X-RateLimit-* headers are now sent on every response. The endpoint docs in src/pages/api/v1/README.md are updated.
  • Extraction is more accurate: content before property now works, an apostrophe in a double-quoted value is kept, and > inside a quoted value is kept.
  • A meta tag longer than 4 KiB is skipped. An unclosed quote swallows what follows, up to 4 KiB, the way a browser would.

Tests

  • npx tsc --noEmit: clean.
  • src/__tests__/og.test.ts: 18 passed. The redirect test now expects 502. New tests cover 405, the 429 on the 21st request with no fetch made, 502 and 504, no error-text leak, attribute order, and 1 MiB of hostile markup.
  • src/__tests__/metaTags.test.ts (new): 15 passed, including four 1 MiB hostile shapes that must parse in under 1 s each.
  • node scripts/run-tests.mjs: CJS 1671 passed and 2 skipped (151 suites); ESM 105 passed (11 suites).
  • A separate review agent read the diff cold. In its first pass the attribute regex was still quadratic per tag (5.4 s per MiB), > inside quoted values was lost, and the perf tests didn't exercise the worst case. All three are fixed, and its re-review found no remaining super-linear input.

Not covered here


Generated by Claude Code

…it, 502/504

GET /api/v1/og is unauthenticated and fetches up to 1 MiB from an
allowlisted host per call. Its per-field regexes
(`<meta[^>]+property=…[^>]+content=…`) rescanned the rest of the page from
every `<meta` opening: 128 KiB of unclosed openings took ~350 ms per regex,
and a 1 MiB page held the event loop for over two minutes across the six
lookups. With OG_ALLOWED_HOSTS=* any page can serve that.

- src/lib/html/metaTags.ts: a single-pass tag/attribute tokenizer (tags
  capped at 4 KiB, quote-aware, resumes where a dropped tag stopped). 1 MiB
  of each hostile shape parses in ~15-40 ms. It also reads content-before-
  property tags, apostrophes in double-quoted values and `>` inside quotes.
- The handler is GET-only (405), rate-limited to 20/min per IP before any
  DNS or fetch, and reports upstream failures as 502 (504 on timeout)
  instead of 500 with the raw internal error message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017rkSXGLxKC5rodwK4bgGgK
@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
multisig Ready Ready Preview Nov 3, 2026 6:11am UTC

Request Review

Copy link
Copy Markdown
Collaborator Author

playwright-browser failed on c68a6af. 1 of 39 tests failed: ring-transfer.spec.ts:339, "ring transfer: sdk → legacy". This isn't caused by this PR's diff.

  • Where it failed: the step at ring-transfer.spec.ts:480, which waits up to 90 s for [data-testid="tx-broadcast-success"] after signer 1 signs.
  • The transaction went through: the Playwright trace for that step shows POST …blockfrost.io/api/v0/tx/submit returned 200 at 06:30:27, followed by transaction.updateTransaction with 200. Only the UI success indicator never became visible to the test, so this looks like a race in the indicator or the assertion, not a failed broadcast.
  • Not in this diff: this PR only changes /api/v1/og and adds a standalone HTML parser. The sign and broadcast path, the transaction card and the e2e helpers are untouched. The app doesn't call /api/v1/og. The only 4xx responses in the trace are 429s from /api/local/og; that endpoint is unchanged here and doesn't block the signing flow.
  • History: the same spec passed on the heads of perf(lookup): share one label-1854 sweep across wallet lookups #430–fix(wallets): stop getUserWallets listing anyone's wallets without a session #433.

Re-run needed: I can't re-run the job from here; the API returned 403. A maintainer can use "Re-run failed jobs" on run 41587997239. If it fails the same way again, I'll treat it as a real failure. No fix for the indicator race exists yet.


Generated by Claude Code

This branch was successfully deployed

1 active deployment
Preview — c68a6afc Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants