Repository navigation
fix(crawl): a verified API key or session bearer is never charged; one fetch reads the page - #665
Merged
Merged
Conversation
…e fetch reads the page x402-gateway 0.16.0 + throttle 0.4.1. The gate now asks crawl-auth.ts first: a ugig API key (Bearer/ApiKey/Token/x-api-key), Supabase access token or AgentPass signature that VERIFIES is never charged; one that is ours and wrong gets 401 instead of the crawl offer. A Chrome UA without Sec-Fetch now reads 20 pages an hour per address before the 402, so an agent reading its own listing once gets it and a bulk crawl still pays. Reported 2026-10-28: GET /api/gigs/<id> and /gigs/<id> with Chrome/129 or Chrome/141 answered 402, and a bogus bearer got 402 rather than 401. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
ThreatCrush Security Scan66 finding(s) HIGH/CRITICAL: 1 | MEDIUM: 12 | LOW: 53
…and 16 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
An agent seller with no session reported
GET /api/gigs/<id>and/gigs/<id>returning the x402 /crawl 402 for Chrome/129 and Chrome/141 UAs. A bogusAuthorization: Beareralso got 402, not 401. A seller with a valid API key could not read its own listing, views or orders.Cause:
chargeSpoofedBrowserscharges any Chrome UA withoutSec-Fetch-Modeon every path, API included, before anything checks who is asking (Chrome/129 also trips the stale rule).Fix
@profullstack/x402-gateway0.16.0 (0.16.0: one fetch reads the page; a site's verified key is never charged x402-gateway#29) + throttle 0.4.1.src/lib/crawl-auth.ts, wired as the gateway'sauthenticatehook. It VERIFIES the credential: a ugig API key (Bearer/ApiKey/Token/x-api-key, checked against the DB), a Supabase access token (GoTrue) or an AgentPass signature. Verified means never charged. A credential that is ours but wrong gets401JSON. A crawl pass (cp_), Power Key or no credential falls through to the normal rules. Only verified results are cached (60s), so a polling integration does not pay bcrypt twice per request.Tests
Full suite 3152+ pass, tsc clean. New
crawl-auth.test.ts(9 cases).crawl-gateway.test.tsnow covers the allowance and the 401 for a bogus bearer.🤖 Generated with Claude Code