diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml index c37bae9c..7cfc10ef 100644 --- a/.github/workflows/pre-commit.yml +++ b/.github/workflows/pre-commit.yml @@ -64,5 +64,7 @@ jobs: pre-commit-${{ runner.os }}- - name: Install pre-commit run: python -m pip install pre-commit + - name: Install markdown linter dependencies + run: npm ci --prefix .github/linters - name: Run pre-commit run: pre-commit run --all-files --show-diff-on-failure --color=always diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index c49200d8..930abfcc 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -110,7 +110,7 @@ repos: entry: bash scripts/test-acq-bats language: system pass_filenames: false - files: '^(acq|acq\.backends/.*|scripts/test-acq-lib\.sh|scripts/test-acq-bats|test/bats/.*)$' + files: '^(acq|acq\.backends/.*|scripts/test-acq-lib\.sh|scripts/test-acq-bats|scripts/fix-exec-bits|scripts/verify-backends|test/bats/.*)$' # Lint the .bats files with ShellCheck (it treats .bats as a shell dialect). # The main shellcheck hook's `types: [shell]` does not reliably tag .bats, @@ -121,3 +121,26 @@ repos: entry: bash -c 'printf "%s\n" "$@" | xargs -r -n8 shellcheck --severity=warning' -- language: system files: '^test/bats/.*\.(bats|bash)$' + + # Guard the executable bit the git index records for entry points. + # + # The repo is routinely checked out onto a container/host shared mount + # (virtiofs, Docker Desktop sharing, WSL drvfs) where the host clone sets + # core.fileMode=false to suppress spurious mode churn. That makes git BLIND to + # a LOST exec bit: a 100755 -> 0644 regression with identical content produces + # an empty `git status` and an empty `git diff --summary`. Since nearly every + # editor/formatter/agent writes a temp file and renames it (new inode, ambient + # umask), entry points silently stop being executable and the next live run + # fails with rc=126 — or, worse, with a misleading downstream error. + # + # --check reports and fails; `scripts/fix-exec-bits` repairs. It compares + # against the INDEX, so sourced libraries (acq.backends/*.sh) and bats files + # stay correctly non-executable; a shebang is not the criterion. + - repo: local + hooks: + - id: fix-exec-bits + name: executable bits match the git index + entry: bash scripts/fix-exec-bits --check + language: system + pass_filenames: false + always_run: true diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4fa80aa0..6d93f4eb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -107,6 +107,7 @@ npm ci --prefix .github/linters | `npm run lint` | Run all linters | | `npm run lint:secrets` | Run gitleaks at the version pinned in `.pre-commit-config.yaml` (pre-commit fetches it; no separate install) | | `npm run check` | Run the full pre-commit suite (gitleaks, shellcheck, YAML/JSON validation, whitespace, markdown lint) | +| `npm run fix:exec-bits` | Restore executable bits the git index records (see below) | > [!NOTE] > `npm run check` auto-fixes some issues (markdown, whitespace, EOF) — review and stage the changes it makes. @@ -118,6 +119,40 @@ kits it applies — and their tests (permission-matrix, model-sync, per-kit repo under `integrations/isolation/acq-kits/`. Changes to provider config, rules, skills, or CA trust belong there. +### If a script suddenly says "Permission denied" + +Run: + +```bash +npm run fix:exec-bits # or: scripts/fix-exec-bits +``` + +Clones on a **container/host shared mount** (virtiofs, Docker Desktop file +sharing, WSL drvfs) do not round-trip POSIX permissions reliably, so such clones +typically set `core.fileMode = false` to stop git reporting a storm of spurious +mode changes. That is the right call, and it creates a blind spot: + +- git applies the index mode on **checkout**, so a fresh clone is correct; +- but almost every editor, formatter, and coding agent saves by writing a temp + file and renaming it. That is a **new inode** with the ambient umask (0644), + so the executable bit is gone; +- with `core.fileMode = false`, git is **blind** to that. A 100755 → 0644 + regression with identical content yields an empty `git status` *and* an empty + `git diff --summary`. + +The symptom is an entry point that stops working with no visible cause — +`./scripts/verify-backends` exiting 126, or the vendored bats failing several +execs deep (`bin/bats` re-execs `libexec/bats-core/bats`, which execs +`bats-exec-suite` and a formatter, so a stripped bit surfaces as a confusing +downstream error rather than an obvious permission one). + +`scripts/fix-exec-bits` compares every tracked file against its **index** mode +and restores the bit. The index is the criterion, not the presence of a shebang: +`acq.backends/*.sh` are *sourced* and `test/bats/*.bats` are read *by bats*, so +both are correctly non-executable. It covers initialized submodules too, and +`--check` makes it report-only (that form runs as a pre-commit hook, and +`scripts/verify-backends` self-heals at startup). + There is one offline unit suite (stubbed `sbx`/`msb`/`opencode`, no Docker or network), built on [bats-core](https://github.com/bats-core/bats-core) (ADR-0025): diff --git a/acq b/acq index 77dc8d6a..7652f047 100755 --- a/acq +++ b/acq @@ -31,6 +31,11 @@ ACQ_RESOLVED_PATH="${ACQ_SCRIPT_DIR}/$(basename "${BASH_SOURCE[0]}")" # shellcheck disable=SC1091 . "${ACQ_SCRIPT_DIR}/acq.backends/common.sh" +# Test-only seam for executable dispatch tests that need a local provider kit. +if [ -n "${BATS_TEST_NAME:-}" ] && [ -n "${ACQ_TEST_USAI_KIT:-}" ]; then + USAI_KIT="$ACQ_TEST_USAI_KIT" +fi + # --------------------------------------------------------------------------- # Print version/identity # --------------------------------------------------------------------------- @@ -107,7 +112,8 @@ Subcommands: sandbox workspace when available. usai-rotate-api-key Rotate the USAi API key. version Show acq version + backend info. - doctor Backend health check + write config. + doctor [NAME] Backend health check, or non-enforcing + image diagnostics for an existing sandbox. backend list List detected backends (sbx, msb). backend set NAME Persist the default backend. backend unset Clear the persisted default (auto-detect). @@ -139,6 +145,10 @@ Global flags (before the subcommand): var/default (a set ACQ_MSB_IMAGE wins, with a notice). The image must satisfy the base-image contract in docs/BACKEND_GUIDE.md. + ACQ_ACTIVATE_PROJECT_ENV=1 + Opt in to using already-approved `direnv export` for + workspaces with .envrc. Devenv-only files are detected and + advised, but acq never runs `direnv allow` or `devenv shell`. run/create flags: (These are recognized only AFTER the subcommand — `acq create --publish …`, not @@ -368,8 +378,10 @@ acq exec — run a command inside a sandbox Usage: acq exec NAME -- CMD... -Runs CMD as the unprivileged agent user in the workspace. On msb, when the host -ssh-agent is forwarded, SSH_AUTH_SOCK is injected so git/ssh reach the agent. +Runs CMD as the unprivileged agent user in the workspace. If the workspace uses +direnv/devenv, acq prints activation guidance; with ACQ_ACTIVATE_PROJECT_ENV=1 it +uses already-approved `direnv export` when .envrc is present. On msb, when the +host ssh-agent is forwarded, SSH_AUTH_SOCK is injected so git/ssh reach the agent. U ;; shell) @@ -379,9 +391,11 @@ acq shell — open an interactive shell in a sandbox Usage: acq shell NAME -Opens an interactive shell in an existing sandbox as the unprivileged agent -user (the human counterpart to 'acq run NAME', which relaunches the recorded -agent; 'acq exec' is non-interactive). On msb, when the host ssh-agent is +Opens an interactive shell in an existing sandbox as the unprivileged agent user +(the human counterpart to 'acq run NAME', which relaunches the recorded agent; +'acq exec' is non-interactive). If the workspace uses direnv/devenv, acq prints +activation guidance; with ACQ_ACTIVATE_PROJECT_ENV=1 it uses already-approved +`direnv export` when .envrc is present. On msb, when the host ssh-agent is forwarded, SSH_AUTH_SOCK is injected so git/ssh reach the agent. U ;; @@ -431,10 +445,14 @@ U ;; doctor) cat >&2 <<'U' -acq doctor — backend health check + write config +acq doctor — backend health check, or sandbox image diagnostics Usage: acq doctor + acq doctor NAME + +With NAME, runs non-enforcing ADR-0030 base-image contract diagnostics inside an +existing sandbox. Warnings are actionable but do not block create/run flows. U ;; *) @@ -516,6 +534,32 @@ _acq_backend_passthrough() { command "${ACQ_RESOLVED_BACKEND}" "$@" } +_acq_secret_service_arg() { + local a1="${1:-}" a2="${2:-}" + case "$a1" in + -g|--global) printf '%s\n' "$a2" ;; + "") printf '\n' ;; + -*) printf '%s\n' "$a1" ;; + *) + case "$a2" in + ""|-*) printf '%s\n' "$a1" ;; + *) printf '%s\n' "$a2" ;; + esac + ;; + esac +} + +_acq_secret_needs_usai_facts() { + [ "$(_acq_secret_service_arg "$@")" = "usai" ] +} + +_acq_secret_import_includes_usai() { + local parsed only_service + parsed=$(_acq_import_parse_args "$@" 2>/dev/null) || return 1 + only_service=$(printf '%s' "$parsed" | cut -f3) + [ -z "$only_service" ] || [ "$only_service" = "usai" ] +} + _acq_parse_secret_scope_for_propagation() { ACQ_PROPAGATE_SCOPE="" ACQ_PROPAGATE_SERVICE="" @@ -928,9 +972,18 @@ case "$subcommand" in ;; doctor) + shift acq_resolve_backend "${explicit_backend}" 2>/dev/null || true - acq_print_doctor - exit 0 + if [ -n "${1:-}" ]; then + if [ -n "${2:-}" ]; then + echo "acq: doctor: expected at most one sandbox name" >&2 + exit 1 + fi + acq_print_image_contract_doctor "$1" + else + acq_print_doctor + fi + exit $? ;; esac @@ -1039,7 +1092,13 @@ case "$subcommand" in # would silently re-pin the host port of every later sandbox in the shell. fi extract_kit_flags "$@" - [ "${#ACQ_CLI_KITS[@]}" -gt 0 ] && _build_kit_list # re-fold with CLI kits + _acq_kit_agent="" + _acq_kit_first=$(first_positional "${ACQ_KIT_REMAINING[@]}" || true) + if [ -n "$_acq_kit_first" ] && is_known_agent "$_acq_kit_first"; then + _acq_kit_agent="$_acq_kit_first" + fi + _build_kit_list "$_acq_kit_agent" + unset _acq_kit_agent _acq_kit_first # Rebuild positional args as: . set -- "$_sub" ${ACQ_KIT_REMAINING[@]+"${ACQ_KIT_REMAINING[@]}"} ;; @@ -1052,6 +1111,7 @@ esac case "$subcommand" in usai-rotate-api-key) shift + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" if ! command -v acq_backend_rotate_key >/dev/null 2>&1; then echo "acq: the '${ACQ_RESOLVED_BACKEND}' backend does not implement key rotation" >&2 echo " (acq_backend_rotate_key). Rotate manually, then re-run." >&2 @@ -1063,6 +1123,7 @@ case "$subcommand" in create) shift + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" acq_backend_prepare # First-run onboarding (ADR-0031): offer `acq configure` once before we # provision, so a brand-new user picks kits up front. No-op after first run @@ -1070,9 +1131,10 @@ case "$subcommand" in maybe_first_run_configure # Apply configured-default extra kits (config.yaml), then offer a create-time # picker pre-populated with those defaults so THIS sandbox can deviate. + create_agent=$(first_positional "$@") _acq_apply_configured_extra_kits || exit 1 create_time_kit_picker - _build_kit_list + _build_kit_list "$create_agent" # Pre-flight: validate the workspace path before provisioning (#issue). ws=$(workspace_path "$@") preflight_workspace_path "$ws" || exit 1 @@ -1095,6 +1157,8 @@ case "$subcommand" in # just proceeds without a scoped token (the token is optional). [ -n "$ws" ] || ws="$PWD" advise_github_scope "$local_name" "$ws" + advise_project_env_activation "$ws" + acq_print_selected_agent_kit "$create_agent" acq_backend_provision "$local_name" "$@" @@ -1127,9 +1191,11 @@ case "$subcommand" in [ "${#create_args[@]}" -gt 0 ] && first=$(first_positional "${create_args[@]}") if [ -n "$first" ] && is_known_agent "$first"; then + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" # AGENT form: create if missing, then attach. local_name=$(derive_name "${create_args[@]}") || exit 1 [ -n "$local_name" ] || { echo "acq: could not determine sandbox name" >&2; exit 1; } + acq_print_selected_agent_kit "$first" # Pre-flight: validate the workspace path before provisioning (#issue). # Only enforce on a fresh create — a re-attach to an existing sandbox @@ -1187,6 +1253,7 @@ case "$subcommand" in # at create; this re-attach case is the live-refeed path. [ -n "$ws" ] || ws="$PWD" advise_github_scope "$local_name" "$ws" + advise_project_env_activation "$ws" else # Fresh create: the USAi API key MUST be stored before we provision, # because a backend (msb) binds secrets only at create time. Creating the @@ -1203,7 +1270,7 @@ case "$subcommand" in maybe_first_run_configure _acq_apply_configured_extra_kits || exit 1 create_time_kit_picker - _build_kit_list + _build_kit_list "$first" if ! ensure_key_present "$local_name"; then exit 1 @@ -1217,6 +1284,7 @@ case "$subcommand" in # proceeds without a scoped token (the token is optional). [ -n "$ws" ] || ws="$PWD" advise_github_scope "$local_name" "$ws" + advise_project_env_activation "$ws" acq_backend_provision "$local_name" "${create_args[@]}" fi @@ -1252,8 +1320,10 @@ case "$subcommand" in warn_if_published_ports_dead "$local_name" if [ "${#agent_args[@]}" -gt 0 ]; then + ACQ_SESSION_KIND="attach" acq_backend_attach "$local_name" -- "${agent_args[@]}" else + ACQ_SESSION_KIND="attach" acq_backend_attach "$local_name" fi @@ -1280,6 +1350,11 @@ case "$subcommand" in acq_backend_ensure_kits_applied "$first" || true maybe_offer_bundle_refresh "${ACQ_RESOLVED_BACKEND}" "$first" "$_pre_heal_status" warn_if_no_ssh_signing_key + _attach_ws="" + if command -v acq_backend_workspace_for >/dev/null 2>&1; then + _attach_ws=$(acq_backend_workspace_for "$first" 2>/dev/null || true) + fi + advise_project_env_activation "$_attach_ws" # Backend-neutral health note (same as the create-and-attach path): if the # sandbox publishes ports with nothing listening behind them, surface it @@ -1304,10 +1379,13 @@ case "$subcommand" in done if [ "${#agent_args[@]}" -gt 0 ]; then + ACQ_SESSION_KIND="attach" acq_backend_attach "$first" -- "${agent_args[@]}" elif [ "${#rest[@]}" -gt 0 ]; then + ACQ_SESSION_KIND="attach" acq_backend_attach "$first" "${rest[@]}" else + ACQ_SESSION_KIND="attach" acq_backend_attach "$first" fi else @@ -1426,6 +1504,12 @@ case "$subcommand" in local_name="${1:-}" [ -n "$local_name" ] || { echo "acq: exec: missing sandbox name" >&2; exit 1; } shift + _exec_ws="" + if command -v acq_backend_workspace_for >/dev/null 2>&1; then + _exec_ws=$(acq_backend_workspace_for "$local_name" 2>/dev/null || true) + fi + advise_project_env_activation "$_exec_ws" + ACQ_SESSION_KIND="exec" acq_backend_run "$local_name" "$@" ;; @@ -1434,6 +1518,12 @@ case "$subcommand" in acq_backend_guard_version local_name="${1:-}" [ -n "$local_name" ] || { echo "acq: shell: missing sandbox name" >&2; exit 1; } + _shell_ws="" + if command -v acq_backend_workspace_for >/dev/null 2>&1; then + _shell_ws=$(acq_backend_workspace_for "$local_name" 2>/dev/null || true) + fi + advise_project_env_activation "$_shell_ws" + ACQ_SESSION_KIND="shell" acq_backend_shell "$local_name" ;; @@ -1458,6 +1548,9 @@ case "$subcommand" in case "$sub2" in set) shift + if _acq_secret_needs_usai_facts "$@"; then + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" + fi acq_backend_secret_set "$@" if _acq_parse_secret_scope_for_propagation "$@"; then _acq_propagate_secret_to_present_backends "$ACQ_PROPAGATE_SERVICE" "$ACQ_PROPAGATE_SCOPE" @@ -1477,6 +1570,9 @@ case "$subcommand" in ;; rm) shift + if _acq_secret_needs_usai_facts "$@"; then + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" + fi # `acq secret rm [-g | SANDBOX] SERVICE` removes an acq-owned secret from # the acq store (and clears the backend's injector). If the args name a # scope + known acq service (-g or a sandbox name), route to the backend @@ -1513,6 +1609,9 @@ case "$subcommand" in # acq_key_injectable so the two can never drift. Fail closed; never forward # to the backend CLI. shift + if _acq_secret_needs_usai_facts "$@"; then + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" + fi # Parse scope the same way secret set/rm do: `-g`/`--global` -> global; a # leading bare token before the service -> sandbox; a lone token -> service. secret_has_scope="" @@ -1588,6 +1687,9 @@ case "$subcommand" in # it never invokes the backend's own `secret` CLI. See acq_secret_import # in acq.backends/common.sh and docs/BACKEND_GUIDE.md "Migration". shift + if _acq_secret_import_includes_usai "$@"; then + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" + fi acq_secret_import "$@" ;; --help|-h|"") diff --git a/acq.backends/agents.sh b/acq.backends/agents.sh index 8da32472..d5532fe9 100644 --- a/acq.backends/agents.sh +++ b/acq.backends/agents.sh @@ -61,3 +61,46 @@ acq_agent_has_msb_install_recipe() { *) return 1 ;; esac } + +# Agent-specific built-in kit metadata. Only the built-in bundle participates in +# implicit selection; team/personal kits supplied through ACQ_EXTRA_KITS or --kit +# are never matched by agent name. +acq_agent_builtin_kit_name() { + case "$1" in + opencode) printf '%s\n' opencode ;; + shell|"") return 1 ;; + *) return 1 ;; + esac +} + +acq_agent_kit_entrypoint() { + case "$1" in + opencode) printf '%s\n' opencode ;; + shell|"") return 1 ;; + *) return 1 ;; + esac +} + +acq_agent_kit_install_owner() { + case "$1" in + opencode) printf '%s\n' kit ;; + shell|"") return 1 ;; + *) return 1 ;; + esac +} + +acq_agent_kit_start_owner() { + case "$1" in + opencode) printf '%s\n' kit ;; + shell|"") return 1 ;; + *) return 1 ;; + esac +} + +acq_agent_builtin_kit_enabled() { + case "$1" in + opencode) return 1 ;; + shell|"") return 1 ;; + *) return 1 ;; + esac +} diff --git a/acq.backends/common.sh b/acq.backends/common.sh index 1f0ee7a0..47cfe23d 100644 --- a/acq.backends/common.sh +++ b/acq.backends/common.sh @@ -36,17 +36,21 @@ PATTERNS_KIT_REPO="git+https://github.com/GSA-TTS/agentic-coding-patterns.git" # the bundle-version anchor recorded in a sandbox's host-side provenance record # (see ACQ_BUILTIN_BUNDLE below + the provenance helpers) so acq can tell a # stale sandbox from a current one. -PATTERNS_KIT_REF="6c6753c60a2b24322fb2e8c0d8e8af60c56ede8f" # agentic-coding-patterns v1.9.0 +PATTERNS_KIT_REF="2dd2ad6ad5f63b842d732424bed5ba9aebeed676" # agentic-coding-patterns v1.10.0 PATTERNS_KIT_DIR="integrations/isolation/acq-kits" -USAI_KIT="${PATTERNS_KIT_REPO}#ref=${PATTERNS_KIT_REF}&dir=${PATTERNS_KIT_DIR}/usai-provider" +USAI_PROVIDER_KIT_NAME="usai-provider" +# shellcheck disable=SC2034 # consumed by _acq_builtin_kit_ref after sourcing +USAI_KIT="${PATTERNS_KIT_REPO}#ref=${PATTERNS_KIT_REF}&dir=${PATTERNS_KIT_DIR}/${USAI_PROVIDER_KIT_NAME}" +# shellcheck disable=SC2034 # consumed by _acq_builtin_kit_ref after sourcing PLAYBOOK_KIT="${PATTERNS_KIT_REPO}#ref=${PATTERNS_KIT_REF}&dir=${PATTERNS_KIT_DIR}/agentic-coding-playbook" +# shellcheck disable=SC2034 # consumed by _acq_builtin_kit_ref after sourcing ZSCALER_KIT="${PATTERNS_KIT_REPO}#ref=${PATTERNS_KIT_REF}&dir=${PATTERNS_KIT_DIR}/zscaler-ca-certificate" +# shellcheck disable=SC2034 # consumed by _acq_builtin_kit_ref after sourcing GITSSHSIGN_KIT="${PATTERNS_KIT_REPO}#ref=${PATTERNS_KIT_REF}&dir=${PATTERNS_KIT_DIR}/git-ssh-sign" # Neutral kit directory names (relative to PATTERNS_KIT_DIR), in apply order. -# kit-translate.sh resolves a kit's spec.yaml + files/ from these names. The -# built-in kit set maps 1:1 to the four *_KIT refs above. +# kit-translate.sh resolves a kit's spec.yaml + files/ from these names. # # ORDER MATTERS: zscaler-ca-certificate is applied FIRST so its CA trust is in # place before any later kit makes an outbound HTTPS request. Behind a @@ -56,7 +60,7 @@ GITSSHSIGN_KIT="${PATTERNS_KIT_REPO}#ref=${PATTERNS_KIT_REF}&dir=${PATTERNS_KIT_ # intercepting CA is already trusted. Establishing trust first makes the rest # of the bundle succeed on corporate networks. # shellcheck disable=SC2034 # consumed by `acq kit list` in the acq entry point -ACQ_KIT_NAMES=(zscaler-ca-certificate usai-provider agentic-coding-playbook git-ssh-sign) +ACQ_KIT_NAMES=(zscaler-ca-certificate "$USAI_PROVIDER_KIT_NAME" agentic-coding-playbook git-ssh-sign) # One-line descriptions for the built-in kits, parallel to ACQ_KIT_NAMES, shown # as the frozen ("always applied") rows in the `acq configure` picker (ADR-0031). @@ -177,6 +181,15 @@ _acq_apply_configured_extra_kits() { # this for a single invocation (parsed in the acq entry point). ACQ_UPDATE_CHECK="${ACQ_UPDATE_CHECK:-1}" +# Human-owned activation contract: acq can detect direnv/devenv workspaces and +# print guidance, but does not evaluate project-provided activation files unless +# the user explicitly opts in for the current invocation/session. +ACQ_ACTIVATE_PROJECT_ENV="${ACQ_ACTIVATE_PROJECT_ENV:-0}" +# In-process marker set only by acq dispatch before user-facing exec/shell/attach. +# Ignore inherited environment values so internal helper probes cannot be wrapped +# by setting ACQ_SESSION_KIND outside acq. +ACQ_SESSION_KIND="" + # Kits supplied on the command line via `--kit ` (repeatable) on # run/create. These are extracted from the arg list by extract_kit_flags (below) # BEFORE the args reach the backend, so a neutral kit ref is translated by acq @@ -184,6 +197,9 @@ ACQ_UPDATE_CHECK="${ACQ_UPDATE_CHECK:-1}" # neutral schema). Folded into the kit list by _build_kit_list, alongside # ACQ_EXTRA_KITS. One ref per element. ACQ_CLI_KITS=() +ACQ_BUILTIN_KIT_COUNT=0 +ACQ_BUILTIN_SUPPORT_KIT_COUNT=0 +ACQ_AGENT_KIT_READY_CACHE="" # Host-port overrides supplied on the command line via `--publish HOST:GUEST` # (repeatable) on run/create, one `HOST:GUEST` pair per element (ADR-0034). @@ -197,9 +213,18 @@ ACQ_PUBLISH_FLAGS=() KIT_SOURCE_PREFIX="github.com/GSA-TTS/" KIT_SOURCE_PREFIXES=("$KIT_SOURCE_PREFIX") -# USAi endpoint constants -USAI_MODELS_URL="https://api.gsa.usai.gov/api/v1/models" -KEY_MGMT_URL="https://gsa.usai.gov/console/key-management" +# Transitional fallback defaults for the pinned usai-provider kit. ADR-0030's +# target state is for the kit to export these provider facts as static metadata +# consumable before sandbox creation. acq keeps these defaults only for backend +# credential binding, key validation, and offline tests until that artifact exists. +# These values are not authoritative once the artifact is available. +USAI_PROVIDER_HOST="api.gsa.usai.gov" +USAI_PROVIDER_BASE_URL="https://${USAI_PROVIDER_HOST}/api/v1" +USAI_PROVIDER_KEY_ENV="USAI_API_KEY" +USAI_PROVIDER_MODELS_URL="${USAI_PROVIDER_BASE_URL}/models" +USAI_PROVIDER_KEY_MGMT_URL="https://gsa.usai.gov/console/key-management" +USAI_PROVIDER_BIND_HOSTS="${USAI_PROVIDER_HOST}" +USAI_PROVIDER_FACTS_SOURCE="fallback" # Source the shared agent catalog (single source of truth for agent tokens and # the sandbox-template image naming convention; issue #377). Both adapters also @@ -291,13 +316,150 @@ ACQ_MANAGED_SECRET_SERVICES=" usai github gitlab " # Only acq-managed services are importable; an unknown service echoes nothing. _acq_import_env_vars_for() { case "$1" in - usai) printf 'USAI_API_KEY\n' ;; + usai) printf '%s\n' "$USAI_PROVIDER_KEY_ENV" ;; github) printf 'GITHUB_TOKEN GH_TOKEN\n' ;; gitlab) printf 'GITLAB_TOKEN\n' ;; *) printf '\n' ;; esac } +_acq_provider_fact_safe_host() { + case "$1" in + ""|*://*|*/*|*..*|*[^A-Za-z0-9.,:-]*) return 1 ;; + *) return 0 ;; + esac +} + +_acq_provider_fact_safe_env() { + case "$1" in + ""|[!A-Za-z_]*|*[!A-Za-z0-9_]*) return 1 ;; + *) return 0 ;; + esac +} + +_acq_provider_fact_safe_url() { + case "$1" in + https://*) ;; + *) return 1 ;; + esac + case "$1" in + *[[:space:]]*|*[[:cntrl:]]*) return 1 ;; + *) return 0 ;; + esac +} + +acq_provider_facts_load() { + local file="${1:-}" line key value schema="" id="" + local host="" base_url="" models_url="" key_env="" key_mgmt_url="" bind_hosts="" + [ -n "$file" ] || return 1 + [ -f "$file" ] || return 2 + + while IFS= read -r line || [ -n "$line" ]; do + case "$line" in ""|'#'*) continue ;; esac + case "$line" in *=*) ;; *) return 1 ;; esac + key="${line%%=*}" + value="${line#*=}" + case "$key" in + ACQ_PROVIDER_FACTS_SCHEMA) schema="$value" ;; + ACQ_PROVIDER_ID) id="$value" ;; + ACQ_PROVIDER_HOST) host="$value" ;; + ACQ_PROVIDER_BASE_URL) base_url="$value" ;; + ACQ_PROVIDER_MODELS_URL) models_url="$value" ;; + ACQ_PROVIDER_KEY_ENV) key_env="$value" ;; + ACQ_PROVIDER_KEY_MGMT_URL) key_mgmt_url="$value" ;; + ACQ_PROVIDER_BIND_HOSTS) bind_hosts="$value" ;; + *) return 1 ;; + esac + done < "$file" + + [ "$schema" = "1" ] || return 1 + [ "$id" = "usai" ] || return 1 + _acq_provider_fact_safe_host "$host" || return 1 + _acq_provider_fact_safe_url "$base_url" || return 1 + _acq_provider_fact_safe_url "$models_url" || return 1 + _acq_provider_fact_safe_url "$key_mgmt_url" || return 1 + _acq_provider_fact_safe_env "$key_env" || return 1 + [ -n "$bind_hosts" ] || bind_hosts="$host" + _acq_provider_fact_safe_host "$bind_hosts" || return 1 + + USAI_PROVIDER_HOST="$host" + USAI_PROVIDER_BASE_URL="$base_url" + USAI_PROVIDER_MODELS_URL="$models_url" + USAI_PROVIDER_KEY_ENV="$key_env" + USAI_PROVIDER_KEY_MGMT_URL="$key_mgmt_url" + # shellcheck disable=SC2034 # consumed by adapters/tests after facts load + USAI_PROVIDER_BIND_HOSTS="$bind_hosts" + # shellcheck disable=SC2034 # consumed by diagnostics/tests after facts load + USAI_PROVIDER_FACTS_SOURCE="$file" +} + +acq_provider_facts_load_from_kit() { + local kitref="${1:-$USAI_KIT}" base_dir="${2:-}" kitdir facts rc + if [ -z "$base_dir" ]; then + base_dir="${ACQ_PROVIDER_FACTS_CACHE_DIR:-${ACQ_STATE_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/acq}/provider-facts}" + fi + mkdir -p "$base_dir" || return 3 + if ! command -v kit_translate_fetch >/dev/null 2>&1; then + return 3 + fi + kitdir=$(kit_translate_fetch "$kitref" "$base_dir/usai-provider") || return 3 + facts="$kitdir/provider-facts/usai.env" + acq_provider_facts_load "$facts" + rc=$? + case "$rc" in + 0) return 0 ;; + 2) return 2 ;; + *) echo "acq: invalid provider facts artifact: $facts" >&2; return 1 ;; + esac +} + +acq_provider_facts_use_fallback() { + USAI_PROVIDER_HOST="api.gsa.usai.gov" + USAI_PROVIDER_BASE_URL="https://${USAI_PROVIDER_HOST}/api/v1" + USAI_PROVIDER_KEY_ENV="USAI_API_KEY" + USAI_PROVIDER_MODELS_URL="${USAI_PROVIDER_BASE_URL}/models" + USAI_PROVIDER_KEY_MGMT_URL="https://gsa.usai.gov/console/key-management" + # shellcheck disable=SC2034 # consumed by adapters/tests after facts load + USAI_PROVIDER_BIND_HOSTS="${USAI_PROVIDER_HOST}" + # shellcheck disable=SC2034 # consumed by diagnostics/tests after facts load + USAI_PROVIDER_FACTS_SOURCE="fallback" +} + +acq_provider_facts_load_from_kit_or_fallback() { + local provider_facts_err provider_facts_rc + if provider_facts_err=$(acq_provider_facts_load_from_kit "$@" 2>&1); then + provider_facts_rc=0 + else + provider_facts_rc=$? + fi + case "$provider_facts_rc" in + 0) : ;; + 1) + [ -z "$provider_facts_err" ] || printf '%s\n' "$provider_facts_err" >&2 + return 1 + ;; + 2) + acq_provider_facts_use_fallback + acq_debug "provider facts unavailable from kit; using transitional fallback defaults" + ;; + 3) + acq_provider_facts_use_fallback + echo "acq: warning: provider facts could not be fetched; using transitional fallback defaults" >&2 + acq_debug "provider facts fetch failed; using transitional fallback defaults" + ;; + *) + [ -z "$provider_facts_err" ] || printf '%s\n' "$provider_facts_err" >&2 + return "$provider_facts_rc" + ;; + esac + # msb derives this adapter-local binding host when sourced, before lazy facts load. + if [ "${ACQ_RESOLVED_BACKEND:-}" = "msb" ]; then + # shellcheck disable=SC2034 # consumed by msb adapter functions at dispatch + ACQ_MSB_USAI_HOST="$USAI_PROVIDER_BIND_HOSTS" + fi + return 0 +} + # _acq_import_detect_var SERVICE -> prints the NAME of the FIRST of SERVICE's # candidate env vars that is set and non-empty; empty output (rc 1) if none is # set. Returns the variable NAME ONLY — never the value — so the secret value is @@ -631,11 +793,158 @@ split_noglob() { eval "$_name=(\"\$@\")" } -# Assemble the full kit list: built-ins, then extras. +_acq_builtin_kit_ref() { + case "${1:-}" in + zscaler-ca-certificate) printf '%s\n' "$ZSCALER_KIT" ;; + "$USAI_PROVIDER_KIT_NAME") printf '%s\n' "$USAI_KIT" ;; + agentic-coding-playbook) printf '%s\n' "$PLAYBOOK_KIT" ;; + git-ssh-sign) printf '%s\n' "$GITSSHSIGN_KIT" ;; + *) return 1 ;; + esac +} + +_acq_builtin_support_kit_names() { + printf '%s\n' zscaler-ca-certificate "$USAI_PROVIDER_KIT_NAME" agentic-coding-playbook git-ssh-sign +} + +_acq_agent_builtin_kit_ref() { + local kit_name + kit_name=$(acq_agent_builtin_kit_name "$1") || return 1 + printf '%s#ref=%s&dir=%s/%s\n' "$PATTERNS_KIT_REPO" "$PATTERNS_KIT_REF" "$PATTERNS_KIT_DIR" "$kit_name" +} + +kit_spec_agent_field() { + local spec="$1" key="$2" + [ -f "$spec" ] || return 1 + awk -v key="$key" ' + function trim(s){ sub(/^[[:space:]]+/,"",s); sub(/[[:space:]]+$/,"",s); return s } + /^agent:[[:space:]]*($|#)/ { in_agent=1; next } + /^[^[:space:]#][^:]*:/ { in_agent=0 } + in_agent { + line=$0 + sub(/[[:space:]]*#.*/,"",line) + if (line ~ "^ " key ":[[:space:]]*") { + sub("^ " key ":[[:space:]]*","",line) + line=trim(line) + gsub(/^\"|\"$/, "", line) + gsub(/^'\''|'\''$/, "", line) + if (line != "") print line + exit + } + } + ' "$spec" +} + +acq_validate_agent_builtin_kit_dir() { + local agent="$1" kitdir="$2" spec schema kind kit_name agent_name entrypoint + local expected_kit expected_entrypoint + expected_kit=$(acq_agent_builtin_kit_name "$agent") || return 1 + expected_entrypoint=$(acq_agent_kit_entrypoint "$agent") || return 1 + spec="$kitdir/spec.yaml" + [ -f "$spec" ] || return 1 + + schema=$(kit_spec_field "$spec" schemaVersion) || schema="" + kind=$(kit_spec_field "$spec" kind) || kind="" + kit_name=$(kit_spec_field "$spec" name) || kit_name="" + agent_name=$(kit_spec_agent_field "$spec" name) || agent_name="" + entrypoint=$(kit_spec_agent_field "$spec" entrypoint) || entrypoint="" + + [ "$schema" = "hybrid/v1" ] || return 1 + [ "$kind" = "mixin" ] || return 1 + [ "$kit_name" = "$expected_kit" ] || return 1 + [ "$agent_name" = "$agent" ] || return 1 + [ "$entrypoint" = "$expected_entrypoint" ] || return 1 +} + +acq_validate_agent_builtin_kit_ref() { + local agent="$1" kitref="$2" base_dir="${3:-}" kitdir + if [ -z "$base_dir" ]; then + base_dir="${ACQ_STATE_DIR:-${TMPDIR:-/tmp}/acq}/agent-kit-validation/$agent" + fi + kitdir=$(kit_translate_fetch "$kitref" "$base_dir") || return 1 + acq_validate_agent_builtin_kit_dir "$agent" "$kitdir" +} + +acq_agent_builtin_kit_ready() { + local agent="$1" kit cache_key cache_value + acq_agent_builtin_kit_enabled "$agent" || return 1 + kit=$(_acq_agent_builtin_kit_ref "$agent") || return 1 + cache_key="${agent}:$(printf '%s' "$kit" | cksum | cut -d' ' -f1)" + cache_value=$(printf '%s\n' "$ACQ_AGENT_KIT_READY_CACHE" | awk -F'\t' -v key="$cache_key" '$1 == key { print $2; exit }') + case "$cache_value" in + yes) return 0 ;; + no) return 1 ;; + esac + if acq_validate_agent_builtin_kit_ref "$agent" "$kit"; then + ACQ_AGENT_KIT_READY_CACHE="${ACQ_AGENT_KIT_READY_CACHE}${cache_key} yes +" + return 0 + fi + ACQ_AGENT_KIT_READY_CACHE="${ACQ_AGENT_KIT_READY_CACHE}${cache_key} no +" + return 1 +} + +_acq_selected_support_kit_refs() { + local name kit + for name in $(_acq_builtin_support_kit_names); do + kit=$(_acq_builtin_kit_ref "$name") || return 1 + printf '%s\n' "$kit" + done +} + +_acq_selected_builtin_kit_refs() { + local agent="${1:-}" kit + _acq_selected_support_kit_refs || return 1 + if [ "${#ACQ_CLI_KITS[@]}" -eq 0 ] && [ -n "$agent" ] \ + && acq_is_known_agent "$agent" && acq_agent_builtin_kit_ready "$agent"; then + kit=$(_acq_agent_builtin_kit_ref "$agent") || return 1 + printf '%s\n' "$kit" + fi +} + +acq_selected_agent_kit_summary() { + local agent="${1:-}" kit_name entrypoint install_owner start_owner apply_state="deferred" + [ "${#ACQ_CLI_KITS[@]}" -eq 0 ] || return 1 + kit_name=$(acq_agent_builtin_kit_name "$agent") || return 1 + entrypoint=$(acq_agent_kit_entrypoint "$agent") || entrypoint="" + install_owner=$(acq_agent_kit_install_owner "$agent") || install_owner="" + start_owner=$(acq_agent_kit_start_owner "$agent") || start_owner="" + acq_agent_builtin_kit_ready "$agent" && apply_state="enabled" + printf 'agent=%s kit=%s entrypoint=%s install_owner=%s start_owner=%s apply=%s\n' \ + "$agent" "$kit_name" "${entrypoint:-none}" "${install_owner:-none}" \ + "${start_owner:-none}" "$apply_state" +} + +acq_print_selected_agent_kit() { + local agent="${1:-}" summary + summary=$(acq_selected_agent_kit_summary "$agent") || return 0 + if [ "${summary##*apply=}" = "enabled" ]; then + printf 'acq: selected built-in agent kit: %s\n' "$summary" >&2 + else + printf 'acq: built-in agent kit candidate (deferred; no agent kit applied): %s\n' "$summary" >&2 + fi +} + +# Assemble the full kit list: selected built-ins, then extras. # Zscaler CA trust FIRST (see ACQ_KIT_NAMES) so later network-fetching kits # succeed behind a TLS-intercepting proxy. +# shellcheck disable=SC2120 # optional agent argument; many callers use default _build_kit_list() { - KITS=("$ZSCALER_KIT" "$USAI_KIT" "$PLAYBOOK_KIT" "$GITSSHSIGN_KIT") + local agent="${1:-}" kit + KITS=() + while IFS= read -r kit; do + [ -n "$kit" ] && KITS+=("$kit") + done < <(_acq_selected_support_kit_refs) + # shellcheck disable=SC2034 # read by adapters/tests after _build_kit_list + ACQ_BUILTIN_SUPPORT_KIT_COUNT="${#KITS[@]}" + if [ "${#ACQ_CLI_KITS[@]}" -eq 0 ] && [ -n "$agent" ] \ + && acq_is_known_agent "$agent" && acq_agent_builtin_kit_ready "$agent"; then + kit=$(_acq_agent_builtin_kit_ref "$agent") || return 1 + KITS+=("$kit") + fi + # shellcheck disable=SC2034 # read by adapters/tests after _build_kit_list + ACQ_BUILTIN_KIT_COUNT="${#KITS[@]}" if [ -n "$ACQ_EXTRA_KITS" ]; then local _extra_kits=() split_noglob _extra_kits "$ACQ_EXTRA_KITS" @@ -811,6 +1120,78 @@ host_path() { printf '%s\n' "$p" } +acq_project_env_kind() { + local ws="${1:-}" + [ -n "$ws" ] && [ -d "$ws" ] || return 1 + if [ -f "$ws/.envrc" ]; then + printf 'direnv\n' + return 0 + fi + if [ -f "$ws/devenv.nix" ] || [ -f "$ws/devenv.yaml" ] || [ -f "$ws/devenv.lock" ]; then + printf 'devenv\n' + return 0 + fi + return 1 +} + +advise_project_env_activation() { + local ws="${1:-}" kind + kind=$(acq_project_env_kind "$ws" 2>/dev/null || true) + [ -n "$kind" ] || return 0 + [ "${ACQ_PROJECT_ENV_NOTICE:-1}" = "0" ] && return 0 + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ]; then + echo "acq: project environment detected ($kind); ACQ_ACTIVATE_PROJECT_ENV=1 will" >&2 + echo " use already-approved direnv export when available." >&2 + return 0 + fi + echo "acq: project environment detected ($kind) in $ws." >&2 + echo " acq will not run project activation, 'direnv allow', or 'devenv shell' automatically." >&2 + echo " If you trust this repo and have approved its .envrc yourself, re-run" >&2 + echo " this acq command with ACQ_ACTIVATE_PROJECT_ENV=1 to use direnv export." >&2 +} + +acq_session_is_user() { + case "${ACQ_SESSION_KIND:-}" in + exec|shell|attach) return 0 ;; + *) return 1 ;; + esac +} + +acq_guest_exec_script() { + printf '%s\n' 'set -e; if [ -n "${ACQ_WORKSPACE:-}" ] && [ -d "$ACQ_WORKSPACE" ]; then cd "$ACQ_WORKSPACE"; fi; if [ -f .envrc ]; then if command -v direnv >/dev/null 2>&1; then eval "$(direnv export sh)"; else echo "acq: .envrc found, but direnv is not installed in this sandbox; continuing without activation." >&2; fi; elif [ -f devenv.nix ] || [ -f devenv.yaml ] || [ -f devenv.lock ]; then echo "acq: devenv files found, but no .envrc/direnv activation is available; continuing without activation." >&2; fi; exec "$@"' +} + +acq_guest_shell_script() { + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ]; then + printf '%s\n' 'set -e; shell="${1:-${SHELL:-/bin/sh}}"; if [ -n "${ACQ_WORKSPACE:-}" ] && [ -d "$ACQ_WORKSPACE" ]; then cd "$ACQ_WORKSPACE"; fi; if [ -f .envrc ]; then if command -v direnv >/dev/null 2>&1; then eval "$(direnv export sh)"; else echo "acq: .envrc found, but direnv is not installed in this sandbox; continuing without activation." >&2; fi; elif [ -f devenv.nix ] || [ -f devenv.yaml ] || [ -f devenv.lock ]; then echo "acq: devenv files found, but no .envrc/direnv activation is available; continuing without activation." >&2; fi; exec "$shell" -l' + else + printf '%s\n' 'set -e; shell="${1:-${SHELL:-/bin/sh}}"; if [ -n "${ACQ_WORKSPACE:-}" ] && [ -d "$ACQ_WORKSPACE" ]; then cd "$ACQ_WORKSPACE"; fi; exec "$shell" -l' + fi +} + +# acq_login_profile_rc_block — emit the guest-side POSIX-sh snippet that a login +# ~/.profile uses to source the kit-owned ~/.rc.d/*.sh drop-ins. Keep this as the +# single backend-neutral source of the bridge text: msb writes it from +# _acq_msb_ensure_agent_shell, and sbx writes the same bytes from +# _acq_sbx_ensure_rc_bridge. +# +# Emitted lines are evaluated by the guest login shell. They gate to bash, +# list snippets with C collation for deterministic order, preserve the readable +# and charset guards, and clean up the loop variable after sourcing. +acq_login_profile_rc_block() { + printf '%s\n' 'if [ -n "$BASH_VERSION" ] && [ -d "$HOME/.rc.d" ]; then' + printf '%s\n' ' for _acq_rc in $(LC_ALL=C ls "$HOME"/.rc.d 2>/dev/null); do' + printf '%s\n' ' _acq_rc="$HOME/.rc.d/$_acq_rc"' + printf '%s\n' ' case "$_acq_rc" in *.sh) ;; *) continue ;; esac' + printf '%s\n' ' [ -r "$_acq_rc" ] || continue' + printf '%s\n' ' case "$_acq_rc" in *[!A-Za-z0-9._/-]*) continue ;; esac' + printf '%s\n' ' # shellcheck disable=SC1090' + printf '%s\n' ' . "$_acq_rc"' + printf '%s\n' ' done' + printf '%s\n' ' unset _acq_rc' + printf '%s\n' 'fi' +} + # _acq_valid_vsock_port PORT — succeed (return 0) iff PORT is an integer in # 1..4294967294 and not the reserved value 123. msb rejects port 0 and reserves # 123 (see ADR-0021); we mirror that validation host-side so a bad port is @@ -1612,16 +1993,18 @@ _acq_provenance_file() { # Write (or overwrite) a sandbox's provenance record. Call ONLY after a # successful bundle apply. Records the bundle identity + the exact applied ref + -# an ISO-8601 UTC timestamp + the backend. Best-effort: a write failure warns -# (debug) and returns non-zero but never aborts the caller (fail-open). -# Usage: acq_provenance_write BACKEND SANDBOX_NAME +# an ISO-8601 UTC timestamp + the backend. A third AGENT argument records the +# agent selected at create time; when omitted, an existing agent field is +# preserved across in-place refreshes. Best-effort: a write failure warns (debug) +# and returns non-zero but never aborts the caller (fail-open). +# Usage: acq_provenance_write BACKEND SANDBOX_NAME [AGENT] [WORKSPACE] acq_provenance_write() { - local backend="${1:-}" name="${2:-}" + local backend="${1:-}" name="${2:-}" agent="${3:-}" workspace="${4:-}" [ -n "$backend" ] && [ -n "$name" ] || return 1 - local file dir ts workspace workspace_source + local file dir ts workspace_source file=$(_acq_provenance_file "$backend" "$name") || return 1 workspace_source=$(acq_provenance_field "$backend" "$name" workspace_source) - if [ "$workspace_source" = "host" ]; then + if [ -z "$workspace" ] && [ "$workspace_source" = "host" ]; then workspace=$(acq_provenance_field "$backend" "$name" workspace) fi dir=$(dirname "$file") @@ -1630,6 +2013,15 @@ acq_provenance_write() { return 1 fi ts=$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo "unknown") + [ -n "$agent" ] || agent=$(acq_provenance_field "$backend" "$name" agent) + case "$agent" in + "") : ;; + *[!a-z-]*) agent="" ;; + esac + case "$workspace" in + "") : ;; + *[!A-Za-z0-9._/-]*) workspace="" ;; + esac # Write atomically via a temp file + mv so a crash mid-write can't leave a # half-written record that later parses as a bogus "current" ref. local tmp="${file}.tmp.$$" @@ -1639,6 +2031,7 @@ acq_provenance_write() { printf 'repo=%s\n' "$ACQ_BUILTIN_BUNDLE_REPO" printf 'applied_ref=%s\n' "$PATTERNS_KIT_REF" printf 'backend=%s\n' "$backend" + [ -n "$agent" ] && printf 'agent=%s\n' "$agent" printf 'applied_at=%s\n' "$ts" if [ -n "${workspace:-}" ]; then printf 'workspace_source=host\n' @@ -2014,11 +2407,12 @@ ensure_opencode_postinstall() { # no free-form error text can splice into the caller's "(HTTP …)" message. check_key() { local name="$1" - local raw code exit_code + local raw code exit_code key_ref + key_ref="\$${USAI_PROVIDER_KEY_ENV}" raw=$(acq_backend_run "$name" -- sh -c \ "curl -sS -o /dev/null -w '%{http_code}' \ - -H \"Authorization: Bearer \$USAI_API_KEY\" \ - $USAI_MODELS_URL; printf '|%s' \"\$?\"" 2>/dev/null || true) + -H \"Authorization: Bearer $key_ref\" \ + $USAI_PROVIDER_MODELS_URL; printf '|%s' \"\$?\"" 2>/dev/null || true) _classify_key_status "$raw" } @@ -2089,11 +2483,12 @@ check_fresh_sandbox_key() { fi # shellcheck disable=SC2064 trap "acq_backend_terminate '$validation_name' /dev/null 2>&1 || true" EXIT - local raw + local raw key_ref + key_ref="\$${USAI_PROVIDER_KEY_ENV}" raw=$(acq_backend_run "$validation_name" -- sh -c \ "curl -sS -o /dev/null -w '%{http_code}' \ - -H \"Authorization: Bearer \$USAI_API_KEY\" \ - $USAI_MODELS_URL; printf '|%s' \"\$?\"" /dev/null || true) + -H \"Authorization: Bearer $key_ref\" \ + $USAI_PROVIDER_MODELS_URL; printf '|%s' \"\$?\"" /dev/null || true) status=$(_classify_key_status "$raw") acq_backend_terminate "$validation_name" /dev/null 2>&1 || true trap - EXIT @@ -2686,7 +3081,7 @@ advise_valid_key() { # the docs rather than guessing the user's network fix. _report_usai_unreachable() { echo >&2 - echo "acq: could not reach the USAi API ($USAI_MODELS_URL) from the sandbox." >&2 + echo "acq: could not reach the USAi API ($USAI_PROVIDER_MODELS_URL) from the sandbox." >&2 echo " The request did not complete (no HTTP response) — this is a network" >&2 echo " reachability problem, NOT an invalid or expired key, so rotating the" >&2 echo " key will not help." >&2 @@ -2713,7 +3108,7 @@ _report_usai_unreachable() { # and point at the docs. _report_usai_unresolved() { echo >&2 - echo "acq: the USAi API host in $USAI_MODELS_URL did not RESOLVE from the sandbox" >&2 + echo "acq: the USAi API host in $USAI_PROVIDER_MODELS_URL did not RESOLVE from the sandbox" >&2 echo " (DNS returned no address). This is a name-resolution problem, NOT an" >&2 echo " invalid or expired key, so rotating the key will not help." >&2 echo " If other public hosts (GitHub, npm) work from the sandbox but only" >&2 @@ -2804,16 +3199,16 @@ ensure_key_present() { # emit a single terse line and fail closed rather than the full interactive # help (mirrors the non-tty guard in the kit-update path above). if [ ! -t 0 ]; then - echo "acq: no USAi API key stored; set one with 'acq secret set -g usai' (see $KEY_MGMT_URL). Aborting." >&2 + echo "acq: no USAi API key stored; set one with 'acq secret set -g usai' (see $USAI_PROVIDER_KEY_MGMT_URL). Aborting." >&2 return 1 fi echo >&2 echo "No USAi API key is stored yet." >&2 - echo "USAi keys are created at $KEY_MGMT_URL and expire every 7 days." >&2 + echo "USAi keys are created at $USAI_PROVIDER_KEY_MGMT_URL and expire every 7 days." >&2 echo >&2 echo "To set one:" >&2 - echo " 1. Open $KEY_MGMT_URL" >&2 + echo " 1. Open $USAI_PROVIDER_KEY_MGMT_URL" >&2 echo " 2. Create a key (or copy an existing one) with the console copy button" >&2 echo >&2 @@ -2888,7 +3283,7 @@ ensure_valid_key() { echo "USAi keys expire every 7 days." >&2 echo >&2 echo "To rotate it:" >&2 - echo " 1. Open $KEY_MGMT_URL" >&2 + echo " 1. Open $USAI_PROVIDER_KEY_MGMT_URL" >&2 echo " 2. Choose 'Rotate' from the Actions menu for your key" >&2 echo " 3. Copy the new key using the console copy button" >&2 echo >&2 @@ -2930,6 +3325,116 @@ ensure_valid_key() { # acq doctor output helpers # ============================================================================ +acq_image_contract_doctor_script() { + cat <<'SH' +ACQ_ADR0030_IMAGE_DIAGNOSTIC=1 +warns=0 +ok() { printf ' ok: %s\n' "$1"; } +warn() { + warns=$((warns + 1)) + printf ' warning: %s\n' "$1" + printf ' fix: %s\n' "$2" +} + +if [ "${HOME:-}" = "/home/agent" ]; then + ok "HOME is /home/agent" +else + warn "HOME is ${HOME:-unset}, not /home/agent" "run diagnostics as the agent user with HOME=/home/agent" +fi + +if [ "$(id -un 2>/dev/null || true)" = "agent" ]; then + ok "running as agent user" +else + warn "diagnostic is not running as the agent user" "run sandbox diagnostics as the unprivileged agent user" +fi + +if [ -d /nix ]; then + ok "/nix exists" + if [ -w /nix 2>/dev/null ]; then + ok "/nix is writable" + else + warn "/nix is not writable" "mount or bake a writable Nix store at /nix for devenv/cache reuse" + fi +else + warn "/nix is missing" "use a devenv-capable image with its Nix store at /nix" +fi + +for tool in nix devenv direnv; do + if command -v "$tool" >/dev/null 2>&1; then + ver=$($tool --version 2>/dev/null | head -n1 || true) + if [ -n "$ver" ]; then + ok "$tool is on PATH ($ver)" + else + ok "$tool is on PATH" + fi + else + warn "$tool is not on PATH" "install $tool in the base image or a create-time kit" + fi +done + +if [ -w /home/agent ]; then + ok "/home/agent is writable" +else + warn "/home/agent is not writable" "make /home/agent owned/writable by the agent user" +fi + +if command -v sudo >/dev/null 2>&1; then + # Best-effort privilege probe: sudo -n may update guest-local sudo/audit state. + if sudo -n true >/dev/null 2>&1; then + ok "passwordless sudo probe works" + else + warn "sudo exists but passwordless sudo probe failed" "grant NOPASSWD sudo for declared privileged kit steps" + fi +else + warn "sudo is not on PATH" "install sudo and allow passwordless sudo where privileged kit features are needed" +fi + +home=${HOME:-/home/agent} +if [ -d "$home/.rc.d" ]; then + ok "~/.rc.d exists" +else + warn "~/.rc.d is missing" "install the neutral shell hook directory at /home/agent/.rc.d" +fi + +hook=0 +for profile in "$home/.profile" "$home/.bashrc" "$home/.zshrc" /etc/profile /etc/bash.bashrc /etc/profile.d/acq*.sh; do + # Match either an explicit `source/. …rc.d…` line OR acq's own login bridge + # marker (acq-login-profile), whose loop sources ~/.rc.d indirectly through a + # variable — so a plain `source .rc.d` grep alone would miss it. See ADR-0030. + if [ -f "$profile" ] \ + && grep -Eq '(^|[[:space:]])(source|\.)[[:space:]].*\.rc\.d|acq-login-profile' "$profile" 2>/dev/null; then + hook=1 + break + fi +done +if [ "$hook" -eq 1 ]; then + ok "shell startup references ~/.rc.d" +else + warn "no shell startup hook for ~/.rc.d found" "source ~/.rc.d snippets from shell startup in deterministic order" +fi + +printf ' summary: %s warning(s); diagnostics only, create/run are not blocked\n' "$warns" +exit 0 +SH +} + +acq_print_image_contract_doctor() { + local name="$1" + if ! command -v acq_backend_doctor_sandbox >/dev/null 2>&1; then + echo "acq: doctor: backend '${ACQ_RESOLVED_BACKEND:-unknown}' cannot inspect sandboxes." >&2 + return 1 + fi + if ! acq_backend_exists "$name"; then + echo "acq: doctor: no such sandbox '$name'." >&2 + return 1 + fi + + printf 'acq: ADR-0030 image contract diagnostics for %s (backend: %s)\n' \ + "$name" "${ACQ_RESOLVED_BACKEND:-unknown}" + echo " This check is non-enforcing: warnings do not change create/run behavior." + acq_backend_doctor_sandbox "$name" +} + acq_print_doctor() { local sbx_status msb_status diff --git a/acq.backends/msb.sh b/acq.backends/msb.sh index ba1129d0..9b0dae66 100644 --- a/acq.backends/msb.sh +++ b/acq.backends/msb.sh @@ -74,6 +74,17 @@ if ! command -v acq_is_known_agent >/dev/null 2>&1; then . "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/agents.sh" fi +_acq_msb_select_kits() { + local agent="${1:-}" + if command -v _build_kit_list >/dev/null 2>&1; then + _build_kit_list "$agent" + elif ! declare -p KITS >/dev/null 2>&1 || [ "${#KITS[@]}" -eq 0 ]; then + KITS=("$ZSCALER_KIT" "$USAI_KIT" "$PLAYBOOK_KIT" "$GITSSHSIGN_KIT") + ACQ_BUILTIN_KIT_COUNT="${#KITS[@]}" + ACQ_BUILTIN_SUPPORT_KIT_COUNT="${#KITS[@]}" + fi +} + # --------------------------------------------------------------------------- # _acq_msb_cli — invoke the msb CLI with MSYS argument rewriting disabled # --------------------------------------------------------------------------- @@ -239,8 +250,11 @@ ACQ_MSB_KIT_CACHE="${ACQ_MSB_KIT_CACHE:-${XDG_CACHE_HOME:-$HOME/.cache}/acq/kits # ACQ_EXEC_READY_TIMEOUT. ACQ_MSB_EXEC_READY_TIMEOUT="${ACQ_MSB_EXEC_READY_TIMEOUT:-${ACQ_EXEC_READY_TIMEOUT:-60}}" -# USAi models path (matches common.sh USAI_MODELS_URL host) for --secret host. -ACQ_MSB_USAI_HOST="api.gsa.usai.gov" +# USAi provider facts for --secret binding; common.sh owns the canonical values. +USAI_PROVIDER_HOST="${USAI_PROVIDER_HOST:-api.gsa.usai.gov}" +USAI_PROVIDER_BIND_HOSTS="${USAI_PROVIDER_BIND_HOSTS:-$USAI_PROVIDER_HOST}" +USAI_PROVIDER_KEY_ENV="${USAI_PROVIDER_KEY_ENV:-USAI_API_KEY}" +ACQ_MSB_USAI_HOST="$USAI_PROVIDER_BIND_HOSTS" # GitHub credential hosts for the msb --secret binding. Bind the REST API and # git-transport hosts so both API calls and HTTPS git clone/push can substitute @@ -283,7 +297,7 @@ _acq_msb_service_binding() { fi fi case "$_service" in - usai) printf '%s\t%s\n' "USAI_API_KEY" "$ACQ_MSB_USAI_HOST"; return 0 ;; + usai) printf '%s\t%s\n' "$USAI_PROVIDER_KEY_ENV" "$ACQ_MSB_USAI_HOST"; return 0 ;; github) printf '%s\t%s\n' "GITHUB_TOKEN" "$ACQ_MSB_GITHUB_HOST"; return 0 ;; esac printf '\t\n' @@ -1031,7 +1045,7 @@ $(acq_secret_meta_list "$_name") EOF fi elif [ -n "${USAI_API_KEY:-}" ]; then - eval "$_arrn+=(--secret \"USAI_API_KEY@\${ACQ_MSB_USAI_HOST}\")" + eval "$_arrn+=(--secret \"${USAI_PROVIDER_KEY_ENV}@\${ACQ_MSB_USAI_HOST}\")" fi } @@ -1084,6 +1098,9 @@ EOF # matter for start. acq_backend_start() { local _name="$1" + if command -v acq_provider_facts_load_from_kit_or_fallback >/dev/null 2>&1; then + acq_provider_facts_load_from_kit_or_fallback "$USAI_KIT" + fi local _start_secret_flags=() _start_secret_names=() _acq_msb_bind_secrets_into _start_secret_flags _start_secret_names "$_name" local _start_rc=0 @@ -3110,20 +3127,12 @@ acq_backend_provision() { # by _acq_msb_vsock_flags_into when an ssh-agent forward is emitted. See ADR-0021. _ACQ_MSB_SSH_AGENT_FORWARDING=0 - # Fetch each built-in kit and gather its create-time contributions. + # Fetch each selected kit and gather its create-time contributions. # Zscaler CA trust FIRST so later network-fetching kits (playbook clone, USAi # validation) succeed behind a TLS-intercepting proxy (e.g. Zscaler). local kitref kitdir - local kits=("$ZSCALER_KIT" "$USAI_KIT" "$PLAYBOOK_KIT" "$GITSSHSIGN_KIT") - # Include any extra kits (env-supplied) and CLI-supplied --kit refs. - if [ -n "${ACQ_EXTRA_KITS:-}" ]; then - local _extras=() - split_noglob _extras "$ACQ_EXTRA_KITS" - kits+=("${_extras[@]}") - fi - if [ "${#ACQ_CLI_KITS[@]}" -gt 0 ]; then - kits+=("${ACQ_CLI_KITS[@]}") - fi + _acq_msb_select_kits "$agent" + local kits=("${KITS[@]}") for kitref in "${kits[@]}"; do kitdir=$(_acq_msb_fetch_kit "$kitref") || { @@ -3808,7 +3817,7 @@ EOF # Record host-side bundle provenance now the built-in bundle is applied. # Best-effort: a provenance write failure never affects the # sandbox. Reached only when provision did not abort earlier under set -e. - acq_provenance_write msb "$name" || true + acq_provenance_write msb "$name" "$agent" || true acq_workspace_record_write msb "$name" "$_first_host" || true # Persist the CLI (`--kit`) and extra (ACQ_EXTRA_KITS) kit refs so a later @@ -4103,13 +4112,15 @@ _acq_msb_ensure_agent_user() { # /bin/sh otherwise — a stock bash-less image keeps working exactly as today. # Also writes a Debian-skel-style ~/.profile bridge: a LOGIN bash reads only # ~/.profile, so without the bridge even `bash -l` skips ~/.bashrc on images -# whose baked home ships no ~/.profile. The bare `export SHELL=/bin/sh` line -# earlier acq versions appended is removed wherever it appears (it stopped a -# login bash cold); a ~/.profile with any other content is left alone (image/ -# user-owned — Debian's own skel already bridges). Idempotent and re-run on -# every provision AND heal (even on an agent-user-ready marker hit), which is -# what upgrades sandboxes created before this setup. Fail-soft: a sync failure -# leaves sessions on the /bin/sh fallback, never blocks the run. +# whose baked home ships no ~/.profile. The bridge also sources kit-owned +# ~/.rc.d/*.sh snippets for bash in deterministic lexical order. The bare +# `export SHELL=/bin/sh` line earlier acq versions appended is removed wherever +# it appears (it stopped a login bash cold); a ~/.profile with any other content +# is left alone (image/user-owned — Debian's own skel already bridges). +# Idempotent and re-run on every provision AND heal (even on an agent-user-ready +# marker hit), which is what upgrades sandboxes created before this setup. +# Fail-soft: a sync failure leaves sessions on the /bin/sh fallback, never blocks +# the run. _acq_msb_ensure_agent_shell() { local name="$1" shell # Probe for bash host-side so the target is decided once and threaded to the @@ -4119,7 +4130,9 @@ _acq_msb_ensure_agent_shell() { shell=$(_acq_msb_safe_shell_or_sh "$shell") acq_debug "msb: syncing agent login shell to $shell in $name" # NOTE: single-quoted `sh -c` string — no single quotes inside (they would - # close the outer quote); `echo` writes the profile lines for the same reason. + # close the outer quote). The rc.d-sourcing block IS single-quote-bearing, but + # it is passed as a positional arg ($2), never interpolated into this string, + # so its quoting is inert here. `echo`/`printf` write the profile lines. # # The bridge exports the shell passwd ACTUALLY holds after the sync attempt # (re-read into `current`), not the requested target: an image with bash but @@ -4127,12 +4140,30 @@ _acq_msb_ensure_agent_shell() { # about the shell sessions really run. # # The rewrite is bounded to a file acq owns outright: missing/empty, or the - # marker present AND still just the bridge (3 lines). Lines other tools - # append below the bridge (rustup et al) must survive the heal, so a - # marker-plus-appended file is left untouched. + # marker present AND still just the bridge (the header lines + shared rc-block + # line count, computed host-side). Lines other tools append below the bridge + # (rustup et al) must survive the heal, so a marker-plus-appended file is left + # untouched. + # The rc.d-sourcing block is the ONE shared bridge text from common.sh + # (acq_login_profile_rc_block); it is threaded to the guest as $2 so the sbx + # and msb bridges cannot silently drift (ADR-0030). The guest writes the 3 + # fixed header lines + the shared block via a single here-doc-free `printf`, + # then re-owns the file to the agent. `_acq_rc_block` is captured host-side so + # the guest receives it as inert data (never re-expanded on the host). + local _acq_rc_block + _acq_rc_block=$(acq_login_profile_rc_block) + # Rewrite bound: 3 fixed header lines + the shared block's line count. Counted + # host-side so the "still just the bridge acq owns" heal guard stays exact if + # the shared block grows (rather than a hard-coded literal that would silently + # go stale). Appended lines below the bridge (rustup et al) push the total + # past this bound and are left untouched. + local _acq_bridge_lines + _acq_bridge_lines=$(( 3 + $(printf '%s\n' "$_acq_rc_block" | wc -l) )) _acq_msb_cli exec "$name" -u 0 -- sh -c ' set -e target="$1" + rc_block="$2" + max_lines="$3" current=$({ getent passwd agent 2>/dev/null || grep "^agent:" /etc/passwd 2>/dev/null; } | head -n1 | cut -d: -f7) if [ "$current" != "$target" ]; then if command -v usermod >/dev/null 2>&1; then @@ -4147,16 +4178,17 @@ _acq_msb_ensure_agent_shell() { if [ -f "$profile" ]; then sed -i "\|^export SHELL=/bin/sh\$|d" "$profile" 2>/dev/null || true fi - if [ ! -s "$profile" ] || { grep -qs acq-login-profile "$profile" && [ "$(wc -l < "$profile")" -le 3 ]; }; then + if [ ! -s "$profile" ] || { grep -qs acq-login-profile "$profile" && [ "$(wc -l < "$profile")" -le "$max_lines" ]; }; then { - echo "# acq-login-profile: written by acq (rewritten on heal; do not edit these 3 lines)." + echo "# acq-login-profile: written by acq (rewritten on heal; do not edit this block)." echo "export SHELL=$current" echo "if [ -n \"\$BASH_VERSION\" ] && [ -f \"\$HOME/.bashrc\" ]; then . \"\$HOME/.bashrc\"; fi" + printf "%s\n" "$rc_block" } > "$profile" _agrp=$(id -gn agent 2>/dev/null || echo agent) chown "agent:${_agrp}" "$profile" fi - ' sh "$shell" /dev/null 2>&1 || { + ' sh "$shell" "$_acq_rc_block" "$_acq_bridge_lines" /dev/null 2>&1 || { echo "acq(msb): warning: could not sync the agent login shell in '$name';" >&2 echo "acq(msb): interactive sessions fall back to /bin/sh." >&2 return 0 @@ -4847,6 +4879,17 @@ EOF _ACQ_MSB_RUN_WS_NAME="" _ACQ_MSB_RUN_WS="" +acq_backend_recorded_agent() { + local name="$1" agent + agent=$(acq_provenance_field msb "$name" agent) + if [ -z "$agent" ]; then + agent=$({ msb exec "$name" -u 0 -- sh -c 'cat /var/lib/acq/agent 2>/dev/null' /dev/null || true; } | tr -d '[:space:]') + fi + if [ -n "$agent" ] && _acq_msb_safe_agent_token "$agent"; then + printf '%s\n' "$agent" + fi +} + # _acq_msb_workspace_for NAME — the guest workspace a session starts in (-w). # Prefer an explicit ACQ_MSB_WORKSPACE override; otherwise the guest path # recorded at provision (it mirrors the host mount path, so it cannot be @@ -4872,6 +4915,10 @@ _acq_msb_workspace_for() { printf '%s\n' "$ws" } +acq_backend_workspace_for() { + _acq_msb_workspace_for "$1" +} + # _acq_msb_term_flags_into ARRVAR — `-e` flags forwarding the host's terminal # identity (TERM, COLORTERM) into an interactive (-t) session. msb does not # propagate the caller's terminal env into the guest PTY, so sessions see @@ -4960,8 +5007,17 @@ acq_backend_run() { _ACQ_MSB_RUN_WS_NAME="$name" _ACQ_MSB_RUN_WS="$ws" fi - _acq_msb_cli exec -u agent -e HOME=/home/agent -w "$ws" ${_sockflag[@]+"${_sockflag[@]}"} \ - ${_gitident[@]+"${_gitident[@]}"} ${_kitenv[@]+"${_kitenv[@]}"} "$name" "$@" + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_session_is_user >/dev/null 2>&1 && acq_session_is_user \ + && command -v acq_guest_exec_script >/dev/null 2>&1 && [ "${1:-}" = "--" ]; then + shift + _acq_msb_cli exec -u agent -e HOME=/home/agent -w "$ws" ${_sockflag[@]+"${_sockflag[@]}"} \ + ${_gitident[@]+"${_gitident[@]}"} ${_kitenv[@]+"${_kitenv[@]}"} \ + -e "ACQ_WORKSPACE=$ws" "$name" -- sh -c "$(acq_guest_exec_script)" sh "$@" + else + _acq_msb_cli exec -u agent -e HOME=/home/agent -w "$ws" ${_sockflag[@]+"${_sockflag[@]}"} \ + ${_gitident[@]+"${_gitident[@]}"} ${_kitenv[@]+"${_kitenv[@]}"} "$name" "$@" + fi } # --------------------------------------------------------------------------- @@ -5059,9 +5115,17 @@ _acq_msb_attach() { local shell shell=$(_acq_msb_agent_passwd_shell "$name") - MSYS2_ARG_CONV_EXCL='*' exec msb exec -t -u agent -w "$ws" ${_term[@]+"${_term[@]}"} -e "SHELL=$shell" \ - ${_sockflag[@]+"${_sockflag[@]}"} ${_gitident[@]+"${_gitident[@]}"} \ - ${_kitenv[@]+"${_kitenv[@]}"} "$name" -- "$agent" "$@" + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_guest_exec_script >/dev/null 2>&1; then + MSYS2_ARG_CONV_EXCL='*' exec msb exec -t -u agent -w "$ws" ${_term[@]+"${_term[@]}"} -e "SHELL=$shell" \ + ${_sockflag[@]+"${_sockflag[@]}"} ${_gitident[@]+"${_gitident[@]}"} \ + ${_kitenv[@]+"${_kitenv[@]}"} -e "ACQ_WORKSPACE=$ws" "$name" -- sh -c \ + "$(acq_guest_exec_script)" sh "$agent" "$@" + else + MSYS2_ARG_CONV_EXCL='*' exec msb exec -t -u agent -w "$ws" ${_term[@]+"${_term[@]}"} -e "SHELL=$shell" \ + ${_sockflag[@]+"${_sockflag[@]}"} ${_gitident[@]+"${_gitident[@]}"} \ + ${_kitenv[@]+"${_kitenv[@]}"} "$name" -- "$agent" "$@" + fi } # _acq_msb_shell_exec NAME [WS] — exec into an interactive login shell as the @@ -5083,9 +5147,17 @@ _acq_msb_shell_exec() { _acq_msb_git_identity_env_flags_into _gitident _acq_msb_kit_env_flags_into _kitenv "$name" _acq_msb_term_flags_into _term - MSYS2_ARG_CONV_EXCL='*' exec msb exec -t -u agent -w "$ws" ${_term[@]+"${_term[@]}"} -e "SHELL=$shell" \ - ${_sockflag[@]+"${_sockflag[@]}"} ${_gitident[@]+"${_gitident[@]}"} \ - ${_kitenv[@]+"${_kitenv[@]}"} "$name" -- "$shell" -l + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_guest_shell_script >/dev/null 2>&1; then + MSYS2_ARG_CONV_EXCL='*' exec msb exec -t -u agent -w "$ws" ${_term[@]+"${_term[@]}"} -e "SHELL=$shell" \ + ${_sockflag[@]+"${_sockflag[@]}"} ${_gitident[@]+"${_gitident[@]}"} \ + ${_kitenv[@]+"${_kitenv[@]}"} -e "ACQ_WORKSPACE=$ws" "$name" -- sh -c \ + "$(acq_guest_shell_script)" sh "$shell" + else + MSYS2_ARG_CONV_EXCL='*' exec msb exec -t -u agent -w "$ws" ${_term[@]+"${_term[@]}"} -e "SHELL=$shell" \ + ${_sockflag[@]+"${_sockflag[@]}"} ${_gitident[@]+"${_gitident[@]}"} \ + ${_kitenv[@]+"${_kitenv[@]}"} "$name" -- "$shell" -l + fi } # --------------------------------------------------------------------------- @@ -5696,26 +5768,15 @@ acq_backend_ensure_kits_applied() { # passwd-shell setup existed. Best-effort like the rest of the heal. _acq_msb_ensure_agent_user "$name" || \ echo "acq(msb): warning: agent-user heal failed for '$name'." >&2 - local kits=("$ZSCALER_KIT" "$USAI_KIT" "$PLAYBOOK_KIT" "$GITSSHSIGN_KIT") - local builtin_count="${#kits[@]}" - if [ -n "${ACQ_EXTRA_KITS:-}" ]; then - local _extras=() - split_noglob _extras "$ACQ_EXTRA_KITS" - kits+=("${_extras[@]}") - fi - # CLI-supplied `--kit ` refs (ACQ_CLI_KITS) MUST be healed too, exactly as - # the provision path folds them in (see acq_backend_provision's kit assembly). - # These kits' STARTUP-phase commands (e.g. openchamber's supervisor loops for - # the shared `opencode serve` and the web UI) are re-run only by this heal — - # `msb start` alone does not replay them (ADR-0017). Omitting them here meant a - # resumed/rebooted sandbox came back with the create-time `-p` port mappings - # intact but NOTHING listening behind them, because the kit's startup was never - # re-run: `acq ports` showed the ports mapped while the services were dead. Fold - # ACQ_CLI_KITS in so `acq run --kit … ` heals its full kit set. - if [ "${#ACQ_CLI_KITS[@]}" -gt 0 ]; then - kits+=("${ACQ_CLI_KITS[@]}") - fi - local ok=1 + # Rebuild after any persisted refs were loaded by the dispatcher. CLI-supplied + # `--kit ` refs must be healed too, exactly as the provision path folds + # them in, so resumed kit services come back with their startup re-run. + local agent + agent=$(acq_backend_recorded_agent "$name") + _acq_msb_select_kits "$agent" + local kits=("${KITS[@]}") + local builtin_count="${ACQ_BUILTIN_KIT_COUNT:-4}" + local kitref kitdir i=0 ok=1 acq_spin_start "Refreshing configuration kits" _acq_msb_heal_kit_set "$name" "$builtin_count" "${kits[@]}" || ok=0 acq_spin_stop "Refreshing configuration kits" @@ -5723,7 +5784,7 @@ acq_backend_ensure_kits_applied() { # msb re-applies all built-in kits idempotently, so on full # success the sandbox carries the currently pinned bundle. Best-effort write. if [ "$ok" -eq 1 ]; then - acq_provenance_write msb "$name" || true + acq_provenance_write msb "$name" "$agent" || true return 0 fi return 1 @@ -6239,6 +6300,11 @@ acq_backend_doctor() { printf '[msb: installed %s]\n' "$ver" } +acq_backend_doctor_sandbox() { + local name="$1" + msb exec -u agent -e HOME=/home/agent "$name" -- sh -c "$(acq_image_contract_doctor_script)" +} + # --------------------------------------------------------------------------- # is_known_agent — used by the acq run dispatch # --------------------------------------------------------------------------- diff --git a/acq.backends/sbx.sh b/acq.backends/sbx.sh index d8718e2c..b89e45e8 100644 --- a/acq.backends/sbx.sh +++ b/acq.backends/sbx.sh @@ -38,6 +38,11 @@ if ! command -v acq_is_known_agent >/dev/null 2>&1; then . "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/agents.sh" fi +USAI_PROVIDER_HOST="${USAI_PROVIDER_HOST:-api.gsa.usai.gov}" +USAI_PROVIDER_BIND_HOSTS="${USAI_PROVIDER_BIND_HOSTS:-$USAI_PROVIDER_HOST}" +USAI_PROVIDER_KEY_ENV="${USAI_PROVIDER_KEY_ENV:-USAI_API_KEY}" +USAI_PROVIDER_MODELS_URL="${USAI_PROVIDER_MODELS_URL:-https://${USAI_PROVIDER_HOST}/api/v1/models}" + # Minimum sbx version required. # # Bumped 0.38.0 -> 0.39.0: acq exports the guest-visible workspace markers @@ -456,6 +461,8 @@ acq_backend_provision() { _acq_sbx_ensure_kit_sources_allowed local name="$1" shift + local agent + agent=$(first_positional "$@") # Host ssh-agent trust-boundary notice (ADR-0021). sbx forwards the host # ssh-agent into the guest IMPLICITLY whenever SSH_AUTH_SOCK is set, so — as on # msb — a user who always exports it (tmux/screen/profile persistence) could @@ -568,9 +575,18 @@ acq_backend_provision() { return 1 fi acq_spin_stop "Waiting for kit startup in '$name'" - acq_provenance_write sbx "$name" || true + acq_provenance_write sbx "$name" "$agent" || true acq_workspace_record_write sbx "$name" "$_primary_ws" || true _acq_sbx_seed_extra_kit_marker "$name" + # Backend parity (ADR-0030): sbx DELIVERS ~/.rc.d/*.sh via kit files[] but, + # unlike msb's .profile bridge, nothing SOURCES them at login. Write the same + # bridge here at create time so a kit-dropped snippet actually runs. This is a + # post-create `sbx exec`, NOT a startup-bearing kit, so it does not interact + # with the sbx >= 0.38 live-extend refusal (that gate only rejects `sbx kit + # add` of setup.startup kits — see _acq_sbx_kit_add). The helper waits for + # exec readiness itself, so this also works when no extra-kit marker write + # happened before it. + _acq_sbx_ensure_rc_bridge "$name" # Persist the CLI (`--kit`) / extra kit refs alongside provenance so a later # resume heal can reload them (see acq_cli_kits_write). Best-effort. acq_cli_kits_write sbx "$name" || true @@ -630,6 +646,82 @@ _acq_sbx_seed_extra_kit_marker() { done } +# --------------------------------------------------------------------------- +# _acq_sbx_ensure_rc_bridge — write the login-profile rc.d sourcing bridge +# --------------------------------------------------------------------------- +# sbx delivers kit-owned ~/.rc.d/*.sh files (via kit files[]), but sbx's agent +# templates ship no ~/.profile that sources them, so on sbx a dropped snippet +# never runs — the exact parity gap this closes (ADR-0030). msb writes an +# equivalent bridge in _acq_msb_ensure_agent_shell; the rc-sourcing block is the +# ONE shared text from common.sh (acq_login_profile_rc_block) so the two +# backends cannot silently drift. The block gates itself to bash and sources +# ~/.rc.d in C-collation (deterministic lexical) order. +# +# Unlike msb, sbx's agent user already has a working login shell, so this ONLY +# adds the rc.d bridge (no shell/passwd sync). It writes ~/.profile only when acq +# owns it outright (missing/empty, or the acq-rc marker present AND still just +# this bridge), so a user/image ~/.profile or lines other tools appended survive +# untouched — the same discipline msb applies. +# +# Runs as a post-create `sbx exec` (NOT a startup-bearing kit), so it is +# unaffected by the sbx >= 0.38 refusal to live-extend a sandbox with +# setup.startup kits (_acq_sbx_kit_add) — no regression to that handling. +# Fail-soft: any error leaves rc.d unsourced (delivery still happened), never +# blocks the run. In offline/no-network mode (ACQ_SBX_KIT_PASSTHROUGH) the real +# `sbx exec` is stubbed by the unit harness, so no live sandbox is required. +_acq_sbx_ensure_rc_bridge() { + local name="$1" rc_block bridge_lines + command -v acq_login_profile_rc_block >/dev/null 2>&1 || return 0 + if ! _acq_sbx_wait_for_exec_ready "$name"; then + echo "acq(sbx): warning: sandbox '$name' not exec-ready; could not install the ~/.rc.d login bridge." >&2 + echo "acq(sbx): kit-dropped ~/.rc.d/*.sh snippets will not be sourced at login." >&2 + return 0 + fi + rc_block=$(acq_login_profile_rc_block) + # Header lines written before the block (the marker comment). Bound computed + # host-side so the "acq owns it outright" guard tracks the shared block's real + # size instead of a literal that could silently go stale if the block grows. + bridge_lines=$(( 1 + $(printf '%s\n' "$rc_block" | wc -l) )) + # The block is threaded to the guest as a positional arg ($1), never + # interpolated into the single-quoted sh -c string (the block contains single + # quotes, which would otherwise close the outer quote). + sbx exec "$name" -- sh -c ' + set -e + rc_block="$1" + max_lines="$2" + profile="$HOME/.profile" + if [ ! -s "$profile" ] || { grep -qs acq-login-profile-rc "$profile" && [ "$(wc -l < "$profile")" -le "$max_lines" ]; }; then + { + echo "# acq-login-profile-rc: written by acq (do not edit this block)." + printf "%s\n" "$rc_block" + } > "$profile" + fi + ' sh "$rc_block" "$bridge_lines" /dev/null 2>&1 || { + echo "acq(sbx): warning: could not install the ~/.rc.d login bridge in '$name';" >&2 + echo "acq(sbx): kit-dropped ~/.rc.d/*.sh snippets will not be sourced at login." >&2 + return 0 + } +} + +acq_backend_recorded_agent() { + local agent + agent=$(acq_provenance_field sbx "$1" agent) + if [ -n "$agent" ] && acq_agent_safe_token "$agent"; then + printf '%s\n' "$agent" + fi +} + +acq_backend_workspace_for() { + acq_provenance_field sbx "$1" workspace +} + +_acq_sbx_attach_command() { + local name="$1" agent + agent=$(acq_backend_recorded_agent "$name") + [ -n "$agent" ] || agent="bash" + printf '%s\n' "$agent" +} + # --------------------------------------------------------------------------- # acq_backend_run — run a command inside a sandbox # --------------------------------------------------------------------------- @@ -641,7 +733,14 @@ acq_backend_run() { # Expect `-- CMD...` separator. No `-u agent` needed: sbx's agent templates # bake the unprivileged `agent` user (UID 1000, HOME=/home/agent) as the # default exec user, unlike a plain msb OCI base (which defaults to root). - sbx exec "$name" "$@" + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_session_is_user >/dev/null 2>&1 && acq_session_is_user \ + && command -v acq_guest_exec_script >/dev/null 2>&1 && [ "${1:-}" = "--" ]; then + shift + sbx exec "$name" -- sh -c "$(acq_guest_exec_script)" sh "$@" + else + sbx exec "$name" "$@" + fi } # --------------------------------------------------------------------------- @@ -652,7 +751,12 @@ acq_backend_run() { # allocates the PTY itself via `exec -it`; exec hands it the terminal directly. acq_backend_shell() { _acq_sbx_apply_git_identity_kit "$1" - exec sbx exec -it "$1" bash + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_guest_shell_script >/dev/null 2>&1; then + exec sbx exec -it "$1" -- bash -lc "$(acq_guest_shell_script)" sh bash + else + exec sbx exec -it "$1" bash + fi } # --------------------------------------------------------------------------- @@ -666,11 +770,32 @@ acq_backend_attach() { echo "acq: refusing to attach to '$name' because kit heal did not finish." >&2 return 1 fi + local agent ws + agent=$(_acq_sbx_attach_command "$name") + ws=$(acq_backend_workspace_for "$name") if [ "$#" -gt 0 ] && [ "$1" = "--" ]; then shift - sbx run --name "$name" -- "$@" + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_guest_exec_script >/dev/null 2>&1; then + if [ -n "$ws" ]; then + sbx run --name "$name" -- env "ACQ_WORKSPACE=$ws" sh -c "$(acq_guest_exec_script)" sh "$agent" "$@" + else + sbx run --name "$name" -- sh -c "$(acq_guest_exec_script)" sh "$agent" "$@" + fi + else + sbx run --name "$name" -- "$@" + fi else - sbx run --name "$name" + if [ "${ACQ_ACTIVATE_PROJECT_ENV:-0}" = "1" ] \ + && command -v acq_guest_exec_script >/dev/null 2>&1; then + if [ -n "$ws" ]; then + sbx run --name "$name" -- env "ACQ_WORKSPACE=$ws" sh -c "$(acq_guest_exec_script)" sh "$agent" + else + sbx run --name "$name" -- sh -c "$(acq_guest_exec_script)" sh "$agent" + fi + else + sbx run --name "$name" + fi fi } @@ -907,6 +1032,29 @@ acq_backend_ensure_kits_applied() { esac fi + if [ "$force" = "1" ] && command -v acq_backend_recorded_agent >/dev/null 2>&1; then + local agent refresh_kit_ref refresh_local refresh_idx support_count handled_support_count label + agent=$(acq_backend_recorded_agent "$name") + _build_kit_list "$agent" + support_count="${ACQ_BUILTIN_SUPPORT_KIT_COUNT:-4}" + handled_support_count=$(_acq_builtin_support_kit_names | wc -l | tr -d ' ') + refresh_idx=0 + for refresh_kit_ref in "${KITS[@]}"; do + refresh_idx=$((refresh_idx + 1)) + [ "$refresh_idx" -le "$handled_support_count" ] && continue + [ "$refresh_idx" -le "${ACQ_BUILTIN_KIT_COUNT:-4}" ] || break + label="agent kit" + [ "$refresh_idx" -le "$support_count" ] && label="support kit" + refresh_local=$(_acq_sbx_translate_kit "$refresh_kit_ref") + _kadd_rc=0; _acq_sbx_kit_add "$name" "$refresh_local" || _kadd_rc=$? + case $_kadd_rc in + 0) echo "acq: $label refreshed in '$name'." >&2 ;; + 3) _acq_sbx_print_recreate_notice "$name"; ok=0 ;; + *) echo "acq: warning: 'sbx kit add' ($label) failed for '$name' (see error above)." >&2; ok=0 ;; + esac + done + fi + _acq_sbx_apply_git_identity_kit "$name" [ "${_ACQ_SBX_LAST_KIT_ADD_SUCCEEDED:-0}" = "1" ] && healed=1 @@ -965,12 +1113,12 @@ acq_backend_ensure_kits_applied() { # needs to know which HOST(s) the credential is injected for and (for the # placeholder/env path) which ENV var. Keep this table backend-neutral here so # sbx.sh and msb.sh agree on the mapping. -# usai -> api.gsa.usai.gov USAI_API_KEY +# usai -> $USAI_PROVIDER_HOST $USAI_PROVIDER_KEY_ENV # github -> github.com,api.github.com GITHUB_TOKEN (sbx built-in service) # Echoes "host1[,host2] ENVVAR"; empty for unknown services. _acq_service_hosts_env() { case "$1" in - usai) printf 'api.gsa.usai.gov\tUSAI_API_KEY\n' ;; + usai) printf '%s\t%s\n' "$USAI_PROVIDER_BIND_HOSTS" "$USAI_PROVIDER_KEY_ENV" ;; github) printf 'github.com,api.github.com\tGITHUB_TOKEN\n' ;; *) printf '\t\n' ;; esac @@ -1570,10 +1718,10 @@ acq_backend_key_present() { local service="${1:-}" scope_sandbox="${2:-}" case "$service" in usai) - if [ -n "$scope_sandbox" ] && _acq_sbx_secret_exists "" "$scope_sandbox" usai USAI_API_KEY; then + if [ -n "$scope_sandbox" ] && _acq_sbx_secret_exists "" "$scope_sandbox" usai "$USAI_PROVIDER_KEY_ENV"; then return 0 fi - _acq_sbx_secret_exists -g "" usai USAI_API_KEY + _acq_sbx_secret_exists -g "" usai "$USAI_PROVIDER_KEY_ENV" ;; *) return 0 @@ -1617,14 +1765,15 @@ _acq_sbx_custom_placeholder() { # --------------------------------------------------------------------------- # acq_backend_rotate_key — rotate the global USAi key (per ADR-0012) # --------------------------------------------------------------------------- -# Rotate the global USAI_API_KEY custom secret in sbx, PRESERVING its proxy +# Rotate the global USAi custom secret in sbx, PRESERVING its proxy # placeholder so existing sandboxes keep resolving to the new value. Carried # verbatim from the former scripts/rotate-apikey (which is now a thin shim that # calls `acq usai-rotate-api-key`). Never places the secret value on argv — sbx # prompts for the new key at its own prompt. Returns non-zero on failure. acq_backend_rotate_key() { - local usai_host="api.gsa.usai.gov" - local usai_models_url="https://${usai_host}/api/v1/models" + local usai_host="$USAI_PROVIDER_HOST" + local usai_models_url="$USAI_PROVIDER_MODELS_URL" + local usai_env="$USAI_PROVIDER_KEY_ENV" # Read the current secret table once (avoids a TOCTOU window + a second call). local secret_ls @@ -1633,25 +1782,25 @@ acq_backend_rotate_key() { return 1 } - # Grab the existing placeholder, anchoring on the USAI_API_KEY token rather + # Grab the existing placeholder, anchoring on the provider key env token rather # than a fixed column, taking only the FIRST match so a duplicated state can't # produce a multi-line value. Preserved across rotation so running sandboxes # that already injected it keep resolving to the new secret. local placeholder placeholder=$(printf '%s\n' "$secret_ls" \ - | awk '{ for (i = 1; i <= NF; i++) if ($i == "USAI_API_KEY") { print $(i+1); exit } }') + | awk -v env="$usai_env" '{ for (i = 1; i <= NF; i++) if ($i == env) { print $(i+1); exit } }') if [ -z "$placeholder" ]; then - echo "acq(sbx): USAI_API_KEY not found. Run 'acq secret ls' to check." >&2 + echo "acq(sbx): $usai_env not found. Run 'acq secret ls' to check." >&2 return 1 fi - # Count USAI_API_KEY rows. More than one means a previous rotation (before the - # fix in ADR-0008) left duplicate/ghost entries; the proxy can then resolve the - # placeholder to an empty entry and validation fails with 401. + # Count provider key env rows. More than one means a previous rotation (before + # the fix in ADR-0008) left duplicate/ghost entries; the proxy can then resolve + # the placeholder to an empty entry and validation fails with 401. local row_count row_count=$(printf '%s\n' "$secret_ls" \ - | grep -cE '[[:space:]]USAI_API_KEY[[:space:]]' || true) + | grep -cE "[[:space:]]${usai_env}[[:space:]]" || true) if [ "${row_count:-0}" -gt 1 ]; then # --- Cleanup path for data corrupted by the pre-fix rotation bug ---------- @@ -1668,25 +1817,25 @@ acq_backend_rotate_key() { # We hold the placeholder already, so remove EACH duplicate row by placeholder # (rm by placeholder targets the one row; loop while any remain), then recreate # a single canonical entry. - echo "Found $row_count USAI_API_KEY entries; consolidating to a single secret." >&2 + echo "Found $row_count $usai_env entries; consolidating to a single secret." >&2 - local _ph="$placeholder" _host="$usai_host" + local _ph="$placeholder" _host="$usai_host" _env="$usai_env" # shellcheck disable=SC2064 trap "{ echo '' >&2 - echo 'ERROR: rotation was interrupted while the USAI_API_KEY secret was removed' >&2 + echo 'ERROR: rotation was interrupted while the USAi secret was removed' >&2 echo 'but before the new value was set. Your sandboxes currently have NO USAi key.' >&2 echo 'Recover by re-running this rotation, or manually:' >&2 - echo ' sbx secret set-custom --host ${_host} --env USAI_API_KEY --placeholder ${_ph}' >&2 + echo ' sbx secret set-custom --host ${_host} --env ${_env} --placeholder ${_ph}' >&2 }" EXIT - # Remove every USAI_API_KEY custom row. Each row's placeholder is re-read from + # Remove every USAi custom row. Each row's placeholder is re-read from # a fresh listing so we clear duplicates that may share OR differ in # placeholder; -f avoids the confirm prompt, /dev/null \ - | awk '{ for (i = 1; i <= NF; i++) if ($i == "USAI_API_KEY") { print $(i+1); exit } }') + | awk -v env="$usai_env" '{ for (i = 1; i <= NF; i++) if ($i == env) { print $(i+1); exit } }') [ -n "$_dup_ph" ] || break if ! rm_err=$(sbx secret rm --placeholder "$_dup_ph" -f &1); then trap - EXIT @@ -1700,7 +1849,7 @@ acq_backend_rotate_key() { # Recreate the single secret with the preserved placeholder. Omitting # --value makes sbx prompt for the new key (keeps it out of shell history). sbx secret set-custom --host "$usai_host" \ - --env USAI_API_KEY --placeholder "$placeholder" || { + --env "$usai_env" --placeholder "$placeholder" || { echo "acq(sbx): 'sbx secret set-custom' failed. See recovery steps above." >&2 return 1 } @@ -1712,7 +1861,7 @@ acq_backend_rotate_key() { # Omitting --value makes sbx prompt for the new key (no shell-history leak). # Global is the default (no `-g`); see the CLI-change note above. sbx secret set-custom --host "$usai_host" \ - --env USAI_API_KEY --placeholder "$placeholder" || { + --env "$usai_env" --placeholder "$placeholder" || { echo "acq(sbx): 'sbx secret set-custom' failed." >&2 return 1 } @@ -1760,7 +1909,7 @@ acq_backend_rotate_key() { local status status=$(sbx exec "$validation_sbx" -- sh -c \ "curl -sS -o /dev/null -w '%{http_code}' \ - -H \"Authorization: Bearer \$USAI_API_KEY\" \ + -H \"Authorization: Bearer \$${usai_env}\" \ $usai_models_url" 2>/dev/null || true) acq_backend_terminate "$validation_sbx" >/dev/null 2>&1 || true @@ -1788,6 +1937,11 @@ acq_backend_doctor() { printf '[sbx: installed %s]\n' "$ver" } +acq_backend_doctor_sandbox() { + local name="$1" + sbx exec "$name" -- env HOME=/home/agent sh -c "$(acq_image_contract_doctor_script)" +} + # --------------------------------------------------------------------------- # is_known_agent — used by the acq run dispatch # --------------------------------------------------------------------------- diff --git a/docs/BACKEND_GUIDE.md b/docs/BACKEND_GUIDE.md index 143fbbb9..c63b2148 100644 --- a/docs/BACKEND_GUIDE.md +++ b/docs/BACKEND_GUIDE.md @@ -1066,6 +1066,24 @@ The neutral vocabulary is: `caps.network.allow`, `files[]`, `commands[]`, `environment`, `publishedPorts`, `volumes`, `agentContext`, `backend_shortcuts`, and `backend_extras`. +**Shell rc snippets.** A kit that needs shell startup behavior should deliver +readable POSIX snippets as files under `/home/agent/.rc.d/*.sh`, for example +`/home/agent/.rc.d/10-team.sh` or `/home/agent/.rc.d/90-personal.sh`. +`acq`'s built-in shell bridge sources these snippets for bash login shells in +deterministic lexical order; zsh-capable base images must wire the same directory +from native zsh startup files. The snippets are kit-owned state, not a +user-editable dotfile layer. + +Use this path for agent shell integration outside a devenv shell, team tool +environment/completions, personal aliases/functions, and an optional +direnv/devenv hook. Do not put secrets in snippets, do not print secret-bearing +environment, and do not duplicate environment already supplied by `devenv shell`. +Fish and nushell do not consume `.sh` files; kits targeting those shells must use +the shells' native configuration directories with the same constraints: kit-owned +files, deterministic names, no secrets, and no duplicate devenv setup. Team kits +that previously installed their own shell-startup loop should migrate their +payloads to `/home/agent/.rc.d/` and stop owning the hook itself. + **`environment` (guest env vars).** A flat map of `NAME → value` for **non-secret** guest environment variables (e.g. `OPENCODE_CONFIG`, `OPENCODE_TUI_CONFIG`, `GITLAB_HOST`). Names must be POSIX identifiers @@ -1146,6 +1164,56 @@ practical floor (`acq kit validate` warns below it). Manage kits with `acq kit list | validate PATH | apply NAME KITREF`. +### ADR-0030 migration sequencing: agent-kit/devenv model + +ADR-0030 moves the `acq run opencode .` implementation toward an agent-kit and +devenv model. The migration is deliberately sequenced so user-visible behavior +stays stable while internals move behind the existing `acq` surface. + +**Increment 0: freeze the current contract before moving code.** Keep the +observable backend-neutral contract green: `--clone` and `ACQ_CLONE=1`, +`ACQ_WORKSPACE`, `ACQ_EXTRA_KITS` before CLI `--kit`, `ACQ_IMAGE`, boot-time +`volumes`, `acq exec` workspace/user behavior, and `acq shell` login-shell +behavior. Verification: + +- `./scripts/test-acq-bats test/bats/140-opencode-migration-gate.bats` +- `./scripts/test-acq-bats test/bats/35-image-override.bats test/bats/116-clone-option.bats test/bats/110-volumes.bats test/bats/30-dispatch-routing.bats test/bats/70-msb-backend.bats` +- `./scripts/verify-backends --only sbx` on an sbx-capable host +- `./scripts/verify-backends --only msb` on an msb-capable host + +**Increment 1: introduce the opencode agent-kit inputs without changing dispatch.** +Add the new kit/devenv artifacts and validation in the patterns/team-kit layer, +but keep `acq run opencode .` wired through the existing wrapper behavior until +Increment 0 remains green. When the team kit ships its own verifier, run it from +that repository or checked-out kit path with `scripts/verify` in addition to the +Quickstart gates above. Unresolved schema or team-kit vocabulary questions stay +inside this increment; do not change `acq` dispatch to depend on them first. + +**Increment 2: switch `opencode` provisioning to consume the agent kit.** Replace +only the internal source of the opencode configuration/devenv setup. Preserve the +same create-time inputs and guest markers. Verification: + +- Increment 0 offline bats commands +- team-kit `scripts/verify` when available +- `./scripts/verify-backends --only sbx` +- `./scripts/verify-backends --only msb` + +**Increment 3: move devenv-specific setup behind the kit boundary.** Remove any +now-duplicated wrapper-side setup only after the kit verifier and Quickstart gates +prove the kit supplies it. Defer unresolved image-layout, login-shell, and +workspace-start-directory questions to this increment so earlier increments do +not overfit speculative behavior. Verification: + +- Increment 0 offline bats commands +- focused live `./scripts/verify-backends --only ` for every backend + whose setup changed +- team-kit `scripts/verify` when available + +**Increment 4: cleanup and documentation.** Only after all gates above pass, +remove obsolete wrapper code and update user docs. Re-run the full offline suite +(`./scripts/test-acq-bats`) and live backend verifier on installed backends before +claiming the ADR-0030 migration complete. + ### Kit-bundle provenance and stale-sandbox refresh `acq` records, host-side, which built-in bundle ref each sandbox was built from diff --git a/docs/CONCEPTS.md b/docs/CONCEPTS.md index 5d795b22..1c282af4 100644 --- a/docs/CONCEPTS.md +++ b/docs/CONCEPTS.md @@ -307,6 +307,66 @@ scheme-less prefix so `acq` allowlists it: export ACQ_EXTRA_KIT_SOURCES="github.com/acme/" ``` +### Advanced: the `~/.rc.d` shell hook + +`acq` gives kits a neutral shell hook: any file a kit drops at +`~/.rc.d/*.sh` is sourced at login. Both backends apply it identically +(ADR-0030) — the delivery is a normal kit `files[]` drop, and `acq` writes the +matching login-profile bridge that sources the snippets (`sbx` writes it at +create time; `msb` writes it as part of its login-shell setup). + +What it is and how it behaves: + +- **Kit-owned, not user-editable.** The snippets come from a kit's `files[]` + tree, and the sourcing bridge is written by `acq`. Treat `~/.rc.d/*.sh` as + build output, not a place to hand-edit inside a sandbox. +- **POSIX-sh, bash bridge.** The built-in `~/.profile` bridge sources the + snippets for bash login shells. Zsh-capable base images must wire the same + directory from native zsh startup files. The files are POSIX-sh. +- **Deterministic lexical order.** Snippets are sourced in byte (C-collation) + order regardless of the guest's locale, so a `10-`, `20-`, `90-` numeric + prefix convention gives a stable, predictable order. A bare shell glob would + sort in the guest's locale collation; the bridge forces `LC_ALL=C` for the + listing to avoid that. + +Guardrails: + +- **Never put secrets in a snippet.** `~/.rc.d/*.sh` is kit content that lands + on disk and is sourced into every login shell; secrets belong in the backend + secret store (injected at runtime), never in a shipped file. +- **Do not duplicate what devenv provides** inside a devenv shell. If a tool's + environment is already established by the project's `devenv`/`.envrc`, do not + re-export it here — the hook is for shell integration that must exist + *outside* a devenv shell. +- **Kit-owned.** Ship shell behavior as a kit, not as ad-hoc edits. + +Use-cases (ADR-0030): + +- the hook itself, delivered by a neutral kit; +- agent shell integration that must exist outside a devenv shell; +- team tool environment variables and shell completions; +- personal aliases and functions (via a personal kit through `ACQ_EXTRA_KITS`); +- a direnv/devenv activation hook. + +**fish and nushell.** The `~/.rc.d/*.sh` snippets are POSIX-sh and are **not** +sourced by fish or nushell, because their shell languages are incompatible with +POSIX-sh syntax — sourcing a `.sh` file into fish/nushell would error, not work. +`acq` therefore does not wire fish/nushell to `~/.rc.d`. This is a documented +constraint, chosen over speculative wiring: fish has its own native autoload +directory (`~/.config/fish/conf.d/*.fish`), and nushell loads from its +configured config/autoload path (`$nu.config-path`). The intended future path is +for a kit to deliver shell-native snippets there when fish/nushell support is +actually needed. Until a concrete need exists, `acq` neither wires those +directories nor installs fish or nushell. + +**Migration path.** If a team kit currently establishes shell behavior by other +means (for example, appending to `~/.bashrc` from a kit `startup` command), move +that behavior into a POSIX-sh snippet under `files/home/agent/.rc.d/NN-name.sh` +in a personal or team kit, applied via `ACQ_EXTRA_KITS`. Pick a numeric prefix +to order it against other snippets (lower runs first). Drop the old +`~/.bashrc`-append step once the snippet is in place; the built-in bridge +sources the snippet for bash login shells. + --- ## Optional Integrations diff --git a/docs/KNOWN_FAILURE_MODES.md b/docs/KNOWN_FAILURE_MODES.md index fa426239..3d34b183 100644 --- a/docs/KNOWN_FAILURE_MODES.md +++ b/docs/KNOWN_FAILURE_MODES.md @@ -2532,6 +2532,182 @@ accepts `0.7.3` or newer. Remove stale copies or adjust `PATH` so the intended --- +## 44. `verify-backends` Stalls Right After "provision complete" (No `DONE` Trace Line) + +**Status:** Fixed — `scripts/verify-backends` captures to a file instead of a pipe. + +### Symptoms + +A live `./scripts/verify-backends --only msb -x` run streams normal progress, +reaches the end of provisioning, and then never returns: + +```text +acq[debug]: msb provision: all kits applied; provision complete (shell-ws-xxxxxx) +acq[debug]: msb: ssh-agent bridge started at /home/agent/.acq/ssh-agent.sock ... +acq[debug]: provenance: recorded msb/shell-ws-xxxxxx applied_ref=... +acq[debug]: cli-kits: recorded msb/shell-ws-xxxxxx (kit= ) +``` + +There is no `[trace ...] DONE create: rc=0` line, no further check output, and no +error. The sandbox itself is healthy — `msb list` shows it running, and +`acq exec -- true` succeeds from another terminal. + +### Root Cause + +The verifier captured each step with command substitution: + +```sh +LAST_OUT=$("$ACQ" "$@" 2>&1 | tee /dev/stderr) +``` + +Command substitution reads the capture pipe until **EOF**, which arrives only +when *every* writer closes it — not when `acq` exits. Two things in `acq` hold a +descriptor on that pipe past exit: + +- `acq.backends/secret-store.sh` dups the process's stderr to fd 9 (the warning + channel used so migration warnings survive `2>/dev/null` read paths). fd 9 is + inherited by children. +- the msb adapter launches **daemonized** guest-side helpers, notably the + ssh-agent `socat` bridge started at the end of provisioning. + +So `acq` finished successfully, while the verifier blocked on a pipe that a +long-lived descendant kept open. The stall therefore appeared at the *last* +thing provisioning logs, making a completed create look like a hung one. + +Because the hang is in the harness, not the backend, it masqueraded as a backend +failure and cost several live runs. Writing to a **file** removes the EOF +dependency entirely: the wait ends when `acq` exits. + +### Fix + +`run_acq` redirects `acq` to a temp file in the hermetic `VERIFY_STATE` dir and +reads the file after `acq` returns. Verbose mode gets its live view from a +`tail -f` on that file rather than from `tee` in the capture pipeline. `acq` stays +in the **foreground**, so pipelines that feed it a secret on stdin +(`printf key | run_acq ... secret set -g usai`) are unchanged. + +### Prevention + +`test/bats/145-verify-capture.bats` extracts the real `run_acq` out of +`scripts/verify-backends` and drives it against a stand-in that daemonizes a +child and exits 0 — the exact shape that deadlocks a pipe capture. Each case runs +under `timeout`, so reintroducing a pipe-EOF dependency fails as a test timeout +instead of hanging a human's live verification. The guard was confirmed to fail +against the old `$( ... | tee /dev/stderr )` implementation before being kept +against the new one. + +### Also Fixed in the Same Pass + +- **`markdownlint` never linted `.github/**/*.md`.** markdownlint's globber skips + dot-directories unless `--dot` is passed, so `.github/pull_request_template.md` + was silently excluded from CI, `npm run lint:md`, and the pre-commit hook. All + three now pass `--dot`. +- **The "did it lint anything?" assertion counted a different set than the + linter.** It hand-translated `.markdownlintignore` into `git ls-files :!` + pathspecs, but those matchers differ: a bare `test/vendor` pathspec excludes + only that exact path, not the tree under it, so the count ran high. The + assertion now evaluates the ignore file with git's own gitignore engine + (`git ls-files -c -i --exclude-from=.markdownlintignore`), which agrees with + markdownlint's semantics. + +### Second Instance of the Same Bug Class + +The grammar-acceptance preflight (`verify_sbx_grammar_acceptance`) had its own +`out=$(bats ... | tee /dev/stderr)` capture — the identical idiom, in a second +place a `run_acq`-only fix would not have reached. Both now go through one shared +`_run_captured` helper, and a test asserts the idiom appears nowhere in the +script, so a third instance cannot be added quietly. + +### The Lost-Executable-Bit Class (and the systemic fix) + +**Status:** Fixed systemically — `scripts/fix-exec-bits`, a pre-commit hook, and +self-healing in `scripts/verify-backends`. + +Three separate symptoms in this one investigation were the same root cause: + +```text +zsh: permission denied: ./scripts/verify-backends +``` + +```text +./scripts/verify-backends: line 259: .../acq: Permission denied # rc=126 +``` + +```text + FAIL sbx: real sbx rejected acq's translated kit grammar + | env: .../libexec/bats-core/bats: Permission denied +``` + +That last one is the dangerous presentation: a **file-permission artifact +reported as kit-grammar incompatibility**. + +#### Why it keeps happening + +A clone on a container/host shared mount (virtiofs, Docker Desktop file sharing, +WSL drvfs) cannot round-trip POSIX permissions, so the clone sets +`core.fileMode = false` to suppress a storm of spurious mode churn. That is +correct, and it opens a blind spot: + +1. git applies the index mode on **checkout**, so a fresh clone is right; +2. nearly every editor, formatter, and coding agent saves by writing a temp file + and renaming it — a **new inode** with the ambient umask (0644), so the exec + bit is dropped; +3. with `core.fileMode = false` git is **blind** to it. Measured: a 100755 → + 0644 drift with byte-identical content yields an empty `git status` *and* an + empty `git diff --summary`. With `core.fileMode = true` the same drift reports + `mode change 100755 => 100644`. + +So the regression is invisible to every normal check and only surfaces as a +confusing runtime failure, often far from the real cause. + +#### Why `bash