Skip to content

Every script the repo runs outside zsh is Python that pytest tests directly #203

Open
0 of 13 issues completed

Description

@ooloth

Current state

Every new test in this repo means choosing between bats and a Python test that drives bash through a subprocess, and a misspelled function name or a call with a missing argument in bash shows up only when that line runs. The repo has about 5,700 lines of bash. All 9 tests outside the shared skills run bash scripts through subprocess, and tools/bats/Brewfile still installs bats. ruff and ty already run from scripts/checks/, pinned in scripts/checks/versions.bash, but there is no pyproject.toml, so each test is its own uv script with its own dependencies.

Ideal state

  • One repo-level Python project holds the dependencies and settings for ruff, ty and pytest, and check runs all three over all of the repo's Python from it
  • Every script the repo runs is Python, apart from the first step of setup, which runs before any language is installed
  • Every test imports the code it tests, and no test runs a bash script
  • The tests run against the checkout they live in without building a throwaway HOME
  • tools/bats/ is archived in tools/@archive/
  • u, symlinks, dcheck and each project command do what they did before their scripts moved to Python
  • Every sub-issue below is closed

Out of scope

  • zsh files, including every shell.zsh

Decisions

Constraints

Open

Starting points

  • scripts/checks/tests.bash: how tests are found and run today, one uv script at a time
  • features/update/tests/test_symlinks.py: a typical test that drives bash through a subprocess against a fake HOME
  • tools/bash/utils.bash: the helpers about 90 scripts source, which Python cannot

Activity

  1. ooloth commented on Oct 30, 2026

    @ooloth
    OwnerAuthor

    Approach

    Port one category at a time after #204. Every crossing between bash and Python is one program starting another, never a function call across languages. The runners already start each step as its own process (bash "$file"), so a ported step is started from the same place its bash version was, and no sourced helper has to cross languages. A sourced helper moves in the same change as its callers. symlink() moves with the link steps (#206), together with its one other caller, tools/pi/update.bash. Before #206 starts, the transition question is settled as "what one command starts the repo's Python", and it covers every caller, not only bash. #206 is the first category to use that command. #208's install steps wait for the bootstrap question, because setup runs them on a fresh Mac. Each category's port takes its tools/@new template file and the add-tool skill with it. After #206 the remaining categories can go in any order, with #210 last.

    What a good fix must achieve 🎯

    Property Category
    P1 After each category moves, u, symlinks, dcheck, check and the project commands give the same results and exit codes as before 🛡️ Safety
    P2 Logic an invariant governs, such as symlink() under DOTFILES_CHECK, has exactly one implementation at every commit 🛡️ Safety
    P3 Each step runs in its own process, so a failing step cannot stop or change the steps after it 🛡️ Safety
    P4 Starting the repo's Python never rewrites uv.lock or installs a package, so it works offline, at Rex's boot and before gcloud is logged in. Checked over every place that starts it 🛡️ Safety
    P5 One command starts the repo's Python, and every caller uses it: zsh aliases, bash scripts, Claude's hook settings, Rex's init.lua and CI. Checked over every caller 🙂 Experience
    P6 Nothing is ported before the decisions it depends on are made, so nothing has to be ported back 🛡️ Safety
    P7 Once a category is Python, no new bash of that kind can be added, and a check fails on a leftover file. Checked over every tool, tools/@new included 🙂 Experience
    P8 The port writes nothing that exists only to be deleted later in the port 🙂 Experience

    Assumptions 🤔

    • A Python step costs about the same to start as a bash step does today. Through uv run --frozen it takes 19 ms. A bash step that sources tools/bash/utils.bash takes 15.5 ms. Both figures are averages of 20 warm runs on this Mac. Across the roughly 60 steps u runs, the difference is about 0.2 s. If Python startup were ten times slower, running the steps in-process would be worth its loss of isolation.
    • symlink() is the only sourced helper that carries an invariant. Outside the link steps, only tools/pi/update.bash calls it. update_and_symlink already starts link steps as a process. The other helpers most scripts source (info, debug, error, warn, have) only print output or check that a command exists, so having a Python copy beside the bash one while both exist is harmless.

    Constraints 🧱

    • Every link.bash also runs on Debian Linux in Amp orbs (docs/invariants/tools.md), so the start command has to work there too.
    • Rex's init.lua starts tools/rex/saver.py at boot without network access. Claude's settings.json runs notify.bash by bare path as a hook.
    • On this Mac, uv run --frozen --no-dev on a project with no runtime dependencies started with no cache and an unreachable package index, whether the project was packaged or not. It needed nothing from the network except a Python. This has not been run on a work laptop.

    Options ⚖️

    Option P1 P2 P3 P4 P5 P6 P7 P8
    🏆 Process boundary, helpers move with their callers, one start command settled before #206 (chosen) ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
    Port helpers first and turn each bash helper into a shim that starts Python ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌
    Once u is Python, it imports each tool's step instead of starting a process ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅
    Sub-issues as written: the first category settles bash-to-Python for the rest ✅ ✅ ✅ ✅ ❌ ❌ ❌ ✅

    Decided by 🧭

    Each option fails on one property. The shims are a CLI that bash calls only while the port runs, and the port's last step deletes it. That is the write-it-twice cost decision 007 gave as its reason for rejecting "not yet". Importing steps in-process lets one tool's sys.exit, os.chdir or environment change reach every tool after it, and the measured gain is about 0.2 s. As written, the sub-issues leave the start command to whichever category goes first, and that question's scope covers only bash callers, not the zsh aliases, the Claude hook or Rex. They also let #208 port install steps before the bootstrap answer says Python exists when setup runs them. #206 goes first because it reaches the most kinds of caller (u, symlinks, dcheck, update_and_symlink, the link-scripts check), and porting it removes the only helper that carries an invariant. That ordering was decided on the spot. It hides no silent harm, it costs no more later, and it closes no option.

    Deferred ⏳

  2. added
    maintenanceUpkeep that changes how the repo is built, not what it does
    on Oct 30, 2026
  3. ooloth commented on Nov 1, 2026

    @ooloth
    OwnerAuthor

    tools/rex/saver.py is gone, so Rex's init.lua no longer starts any of the repo's Python at boot. Rex's own server now restores sessions after a restart (decision 009). The constraint above that Python starts at Rex's boot without network access no longer applies, and P5's callers no longer include Rex's init.lua.

  4. added a commit that references this issue on Nov 1, 2026
  5. ooloth commented on Nov 5, 2026

    @ooloth
    OwnerAuthor

    The new auth and secret aliases are callers of the repo's Python, started with uv run --offline <path>. auth exists to run before the Google credentials are valid, so whatever start command is chosen has to work offline with expired work-index credentials. docs/questions/what-command-starts-the-repos-python.md lists both as callers.

  6. ooloth commented on Nov 19, 2026

    @ooloth
    OwnerAuthor

    tools/claude/cc.bash now has a caller outside this repo: the work laptop's hub hub.toml names it by absolute path as [investigation] launcher (ooloth/hub#435). hub runs that path directly with its own arguments, and refuses to start when it isn't a file with an execute bit. A port has to leave a directly executable file at that path, or change that hub.toml in the same change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintenanceUpkeep that changes how the repo is built, not what it does

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions