astral-html

Warning

This project is an experiment in developing a library using Codex. The content below the line, as well as all the code, are written by Codex.

---

A very fast HTML parser with CSS selectors for Python web scrapers. It’s written in Rust and uses the astral-html tokenizer crate.

Each parser reads a saved page and extracts the URLs and text of links matching a page-specific CSS selector. Bars show mean time per scrape across 12 fresh processes. Whiskers show one standard error, and each page has its own scale. Three-snapshot results · 1,867-page corpus results

Quick start

Pass the page HTML as a string, then select the links that have an href.

from astral_html import parse

html = """
<main>
    <a href="/guide">Read <b>the guide</b></a>
    <a>Draft</a>
</main>
"""
doc = parse(html)

links = []
for link in doc.select("main a[href]"):
    href = link["href"]
    links.append((href, link.text))

print(links)  # [('/guide', 'Read the guide')]

The tree follows nesting in the source. For example, in <ul><li>A<li>B</ul> the second li is a child of the first, so select("ul > li") returns one item with the text "AB".

Parsing and CSS queries · Development guide

Licensed under MIT or Apache-2.0.