Repository navigation
Conversation
…it, 502/504 GET /api/v1/og is unauthenticated and fetches up to 1 MiB from an allowlisted host per call. Its per-field regexes (`<meta[^>]+property=…[^>]+content=…`) rescanned the rest of the page from every `<meta` opening: 128 KiB of unclosed openings took ~350 ms per regex, and a 1 MiB page held the event loop for over two minutes across the six lookups. With OG_ALLOWED_HOSTS=* any page can serve that. - src/lib/html/metaTags.ts: a single-pass tag/attribute tokenizer (tags capped at 4 KiB, quote-aware, resumes where a dropped tag stopped). 1 MiB of each hostile shape parses in ~15-40 ms. It also reads content-before- property tags, apostrophes in double-quoted values and `>` inside quotes. - The handler is GET-only (405), rate-limited to 20/min per IP before any DNS or fetch, and reports upstream failures as 502 (504 on timeout) instead of 500 with the raw internal error message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017rkSXGLxKC5rodwK4bgGgK
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Collaborator
Author
|
Re-run needed: I can't re-run the job from here; the API returned 403. A maintainer can use "Re-run failed jobs" on run 41587997239. If it fails the same way again, I'll treat it as a real failure. No fix for the indicator race exists yet. Generated by Claude Code |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
GET /api/v1/ogis public and unauthenticated, and each call fetches up to 1 MiB from an allowlisted host. This PR bounds the endpoint:<meta[^>]+property=…[^>]+content=…) rescanned the rest of the page from every<metaopening. They are replaced bysrc/lib/html/metaTags.ts, a single-pass tag and attribute tokenizer. It caps each tag at 4 KiB, ignores>inside quoted values, and resumes from where a dropped tag stopped.Allow: GET.Upstream 404,connect ECONNREFUSED …) is no longer echoed to the caller; unexpected errors are only logged.Why
The regexes are quadratic in the number of unclosed
<metaopenings:<metaopenings, one regex=<meta a="x>(unclosed quotes)All of that time is spent on the event loop of a public endpoint. With the default allowlist an attacker needs to control the HTML on an allowlisted host. With the supported
OG_ALLOWED_HOSTS=*setting, any page they host is enough.Behaviour changes callers will notice
X-RateLimit-*headers are now sent on every response. The endpoint docs insrc/pages/api/v1/README.mdare updated.contentbeforepropertynow works, an apostrophe in a double-quoted value is kept, and>inside a quoted value is kept.Tests
npx tsc --noEmit: clean.src/__tests__/og.test.ts: 18 passed. The redirect test now expects 502. New tests cover 405, the 429 on the 21st request with no fetch made, 502 and 504, no error-text leak, attribute order, and 1 MiB of hostile markup.src/__tests__/metaTags.test.ts(new): 15 passed, including four 1 MiB hostile shapes that must parse in under 1 s each.node scripts/run-tests.mjs: CJS 1671 passed and 2 skipped (151 suites); ESM 105 passed (11 suites).>inside quoted values was lost, and the perf tests didn't exercise the worst case. All three are fixed, and its re-review found no remaining super-linear input.Not covered here
/api/local/og, the endpoint the UI actually calls, has regexes of the same shape. It is not touched here because it is in flight in fix(dapps): only proxy images, and stop following redirects off the allow-list #424. Once fix(dapps): only proxy images, and stop following redirects off the allow-list #424 lands it can switch toreadMetaTags.getClientIPtrusts the firstX-Forwarded-Forentry, so the per-IP limit depends on the platform overwriting that header. That behaviour predates this PR and is unchanged.OG_ALLOWED_HOSTSvalue.Generated by Claude Code