Skip to content

fix: count sites instead of hostnames in public_hash_list - #357

Open
tomayac wants to merge 1 commit into
HTTPArchive:mainfrom
tomayac:public-hash-list-sites
Open

tomayac wants to merge 1 commit into
HTTPArchive:mainfrom
tomayac:public-hash-list-sites

Conversation

@tomayac

@tomayac tomayac commented Sep 29, 2026

Copy link
Copy Markdown
Member

The ≥100 threshold in public_hash_list counts distinct hostnames of the resource URL, so a font used only on 100 $city.$vulnerable-group.com landing pages passes, even though it identifies a single site, and any origin could then probe Cross-Origin Storage for its hash to infer that the user belongs to that group. This PR counts distinct sites (eTLD+1 of the embedding page) using the full Public Suffix List, including the private section that BigQuery's NET.PUBLIC_SUFFIX() skips, so alice.github.io and bob.github.io still count as two sites, renames num_origins to num_sites, and adds a script plus a monthly workflow that keep the private suffix rules current. @max-ostapenko, could you please review?

The >=100 threshold counted distinct hostnames of the resource URL, so a
resource used only on many subdomains of one site (for example
$city.$vulnerable-group.com) passed the privacy gate. Count distinct
sites (eTLD+1 of the embedding page) using the full Public Suffix List,
including the private section that BigQuery's NET.PUBLIC_SUFFIX() skips.
Add a script and a monthly workflow to keep the private rules current.
@tomayac

tomayac commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

This is a follow-up to #324.

@tunetheweb

Copy link
Copy Markdown
Member

TIL NET.REG_DOMAIN only considers ICANN domains from the PSL and not Private PSL domains! That's sad... 😔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants