Skip to content

Every Python script turns what it reads into domain types before acting on it #248

Open

Description

@ooloth

Current state

A field that GitHub, a package registry or a subprocess renames or leaves out reaches the middle of most of the repo's Python scripts as a missing key or an Any, instead of stopping the script at its input with a sentence that names the problem. Of the 13 scripts the repo runs, 9 read JSON from gh, a registry or a subprocess, and 7 of those parse it without Pydantic. Any appears in 6 of them, most in review_dependency_bump.py, and 9 read their arguments from an argparse.Namespace. Only merge.py and capture_window.py use NewTypes for domain values.

Ideal state

  • Every script the repo runs turns its command-line arguments and each source of data it reads into domain types in one parse step per source, as the boundary Must in python.md describes
  • No value typed Any, no raw dict and no argparse.Namespace reaches code past that step, and ty confirms it
  • A script given malformed input ends with a sentence naming every problem it found, before it changes anything
  • Every script parses its command-line arguments the way the decision record below settles

Decisions

Constraints

Open

  • How do the repo's scripts parse command-line arguments: argparse read only inside a typed parse function, or a library whose result the type checker knows, such as tyro or Typer? (new external dependency, and a CLI pattern every script follows)

Starting points

  • tools/agents/config/standards/python.md: the boundary Must the scripts are measured against
  • tools/agents/config/skills/review-code/scripts/review_dependency_bump.py: the script with the most Any and raw dict values today
  • tools/agents/config/skills/prove-it-works/scripts/capture_window.py: the one script that already parses both its arguments and its subprocess output into domain types

Activity

  1. ooloth commented on Nov 3, 2026

    @ooloth
    OwnerAuthor

    Measurements for the open question

    Measured on a personal Mac. Each import time is the median of 12 runs of import x, beyond starting Python.

    Option Import time Packages installed What the code receives Errors
    argparse 4 ms none (standard library) an argparse.Namespace, typed Any stops at the first
    Typer 0.27.3 30 ms 7, including rich typed function parameters stops at the first
    tyro 1.0.16 37 ms 4 the frozen dataclass it was given, NewType fields intact no hook for a check of our own
    pydantic-settings 2.16.2 CliApp 90 ms 7 a typed model reports every error at once
    • pydantic-settings prints a traceback by default. When --app was missing, it read APP=Safari from the environment without saying so.
    • tyro's API for building a custom type from a string was awkward to use. A plain dataclass with NewType fields worked.
    • Pydantic alone imports in 27 ms.
    • Cyclopts was not tried.
  2. ooloth commented on Nov 3, 2026

    @ooloth
    OwnerAuthor

    argparse cannot report every problem at once

    Found while reworking capture_window.py, which reads its arguments with argparse inside one typed parse function, the argparse option this ticket leaves open.

    • What the parse function collects: every problem in a value argparse hands over, such as a blank --app, an --out that isn't .png, a --window-id that isn't a positive number, or --title given with --window-id. With no flag marked required, missing flags are collected too. One run reported four problems together.
    • What it cannot collect: a flag argparse cannot read at all, such as an unknown flag or one missing its value. argparse exits 2 on the first of these, before the parse function runs, so any other problems in the same command go unreported.
    • What this means here: with plain argparse, the boundary rule in python.md that a parse "reports every problem it finds at once" holds for values, but not for malformed flags.
  3. added
    maintenanceUpkeep that changes how the repo is built, not what it does
    on Nov 5, 2026
  4. ooloth commented on Nov 5, 2026

    @ooloth
    OwnerAuthor

    features/auth/auth.py and features/secret/secret.py are two more scripts the repo runs. Both use NewTypes and frozen-dataclass unions, and auth.py parses argocd's JSON with json.loads, without Pydantic. Both are stdlib-only and run with uv run --offline, so auth can start while the Google credentials the work package index needs have expired. A parsing library added to every script would have to keep that working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintenanceUpkeep that changes how the repo is built, not what it does

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions