Repository navigation
feat(oci-engine): add rootless podman OCI-engine capability kit - #437
Conversation
Add the patterns-side `oci-engine` neutral acq mixin kit (schemaVersion "hybrid/v1", kind mixin, name oci-engine) under integrations/isolation/acq-kits/oci-engine/. It provides a rootless container engine (podman) as a neutral, opt-in capability, implementing the isolation ADR-0030 decision to move that provisioning out of the msb backend adapter (sibling agentic-coding-quickstart repo) and into a backend-agnostic kit. acq applies it only when the operator opts in via ACQ_ENABLE_OCI_KIT on the quickstart side. The kit ports the working msb-adapter logic into the neutral vocabulary: an install-phase script (root, create-time once) installs podman + the rootless prerequisites, selects a storage driver that works on an overlay root (overlay+fuse-overlayfs, else vfs), writes a Docker-Hub-first registry config, and shadows the dead-socket base docker CLI with a docker->podman wrapper; a startup-phase script (root, every start) re-grants /dev/net/tun + /dev/fuse to the agent because /dev is a devtmpfs re-created each boot. Both scripts are idempotent, POSIX sh, and fail soft. Preserves the ADR-0020 security decisions: the engine runs rootless as the agent (never root; only install/config needs root), no docker-in-docker daemon lifecycle, explicit group-scoped (0660, not world-writable) device handling, and Docker-Hub-first registry resolution with short-name-mode="enforcing" as the DEFAULT (least-privilege / prompt-injection defense) rather than permissive. Declares no caps.network.allow: the OS package-mirror host depends on the base distro and is already reachable under acq's balanced egress baseline (ADR-0002), so hardcoding a distro mirror would be wrong and redundant; a strict tier fails the install soft. Ships a host-side scripts/verify live check (rootless podman build FROM scratch + docker->podman wrapper), skipped in CI. Registers the kit in kits.yaml with a prose parity note. Co-authored-by: OpenCode Agent <agent@gsa.gov>
…ess verify Address three findings from adversarial review of the oci-engine kit. Fix 1 (blocker) — make the install phase FAIL SOFT. `set -eu` plus bare apt-get/dnf/apk calls aborted the install script on the first non-zero return, which fails `sbx create` (a dead sandbox) — the opposite of the documented fail-soft promise. Run each distro install inside a subshell under its own `set -e` and swallow the exit; a failure now falls through to the single warn-and-exit-0 `command -v podman` re-check so provision continues. Verified by stubbing failing package managers on PATH: the script prints the warning and exits 0. Fix 2 — restore the reference adapter's rootless recovery. Add a second startup step run as the agent (uid 1000), files/home/oci-engine-vfs-fallback.sh: a real rootless layer-mount self-test (podman build FROM scratch) and, on failure, a user-level ~/.config/containers/storage.conf selecting vfs, then retry. Idempotent and fails soft (always exits 0). Covers bases whose overlay+fuse-overlayfs combo is rejected under rootless, where `podman info` passes but a layer mount fails. Fix 3 — make scripts/verify actually prove rootless. Run the agent-phase checks with an explicit `-u agent` (matching the reference), and hard-fail step 4 when podman does not report rootless=true (falling back to a non-zero agent uid assertion when the format key is unavailable) instead of printing PASS regardless. Also add oci-engine/docs/decisions/ recording the ADR-0030 port and the fail-soft + vfs-fallback behavioral choices, matching the other kits. Co-authored-by: OpenCode Agent <agent@gsa.gov>
…dapter OCI off Rework the kit's host-side live check so it demonstrates the kit — not the msb adapter's legacy auto-install — actually provisions rootless podman. - Switch from raw `sbx create --kit` to the real path: `acq --backend msb create shell <ws> --kit <this kit>` + `acq exec` (which runs as the agent user), exercising kit-translate + kit apply end-to-end. msb-only, since msb is acq's default backend and the backend that demonstrates OCI behavior (ADR-0020); sbx base-image selection is out of scope. - Export ACQ_MSB_ENSURE_OCI=0 for the run so acq's still-active adapter auto-install cannot mask a broken kit with a false pass. With it off, podman and especially the /usr/local/bin/docker wrapper are attributable to the kit. When the adapter auto-install is removed (ADR-0030), the export becomes a no-op and this stays the standing regression test. - Derive the sandbox name the way acq does (source acq under ACQ_SOURCE_ONLY, call derive_name) for correct exec/cleanup, with a slug fallback. - Keep the rootless assertions (rootless=true, else non-zero agent uid) and the FROM-scratch layer-mount build. Update README's Verifying section to match. Companion quickstart work (add ACQ_ENABLE_OCI_KIT gate, retire the adapter's OCI auto-install, repoint verify-backends) is tracked in the quickstart repo. Co-authored-by: OpenCode Agent <agent@gsa.gov>
…an info stderr msb's exec transport returns guest stdout as CRLF, so `in_sbx` captured a trailing \r that (a) broke exact-string compares — e.g. "/usr/local/bin/docker\r" != "/usr/local/bin/docker", a false FAIL in the docker-wrapper check — and (b) when printed, moved the cursor to column 0 so the next char overwrote the line (the garbled ") PASS" / ") FAIL" output). Strip \r once in in_sbx (the same fix the quickstart's run_acq uses). Also surface podman's actual stderr when the step-4 `podman info` check fails, so a failure is diagnosable instead of a black box. Co-authored-by: OpenCode Agent <agent@gsa.gov>
…ves the kit Two changes so verify actually proves the KIT provisions podman, and fails legibly when it does not: - Pin a neutral base image that ships neither docker nor podman (VERIFY_OCI_IMAGE, default docker/sandbox-templates:shell) via acq's --image. The msb default base (…:shell-docker) already bundles a container engine and puts the agent in the docker group, so podman/docker present there is NOT attributable to the kit — defeating ACQ_MSB_ENSURE_OCI=0. On a neutral base the kit is the only thing that can produce a working rootless podman (it installs from the OS mirror at create). Override with VERIFY_OCI_IMAGE=<ref>. - Make "podman present" a HARD GATE (step 3): if the kit's fail-soft install did not land podman, dump the oci-engine lines from the create log (the real reason — unreachable mirror, unsupported base) and stop, instead of cascading into confusing rootless FAILs. This is what the last run needed: on …:shell-docker the base had no engine the kit installed, the fail-soft install exited 0, and podman was silently absent. README Verifying section updated to document the neutral-base requirement and the VERIFY_OCI_IMAGE override. Co-authored-by: OpenCode Agent <agent@gsa.gov>
…om install Live testing on a base that already bundles a container engine (docker/sandbox-templates:*-docker runs a real dockerd) surfaced two issues in the install script; fix both. - Functional-engine early-out: before doing anything, if `docker info` succeeds the base already has a WORKING engine, so leave it alone (do not install podman or shadow the working docker) and exit 0. Probe `docker info` (function), NOT merely a docker BINARY: many bases ship a docker CLI whose daemon socket is dead in the sandbox (the exact case this kit fixes), which must NOT early-out and correctly proceeds to install podman. - Decouple config/wrapper from the package step: the storage/registry config and the docker->podman wrapper now run whenever podman is PRESENT at that point (installed by the kit OR pre-existing), not only when the kit ran the package install. Previously a skipped/failed package step also skipped the config, which is why /usr/local/bin/docker was absent on a base without podman. Both paths keep the fail-soft posture (exit 0). Verified via stubbed shells: no-engine+failing-mirror -> fail-soft; functional docker -> early-out; dead-socket docker -> proceeds to install. spec.yaml, README, and the kit's docs/decisions updated to describe the early-out and the decoupling. Co-authored-by: OpenCode Agent <agent@gsa.gov>
|
AI-assisted adversarial review (nexus-agents Architecture is sound (Scope Steward did a build-vs-buy pass: right layer per ADR-0030, podman/fuse-overlayfs/passt/slirp4netns are the correct bought components, not a reimplementation, and this consolidates provisioning that's currently duplicated in the msb adapter — net DRY-positive). The blocking issues are all in the implementation's verifiability and a few concrete correctness/security defects, confirmed independently by multiple voters with line citations: 1. Fail-silent, not fail-soft (all 5 voters, highest severity). Every failure path in 2. Real supply-chain gap: package-manager flag injection (Security, DevEx, Architect). 3. Unconditional elevated device grant (Security, DevEx, catfish). 4. Storage driver permanently mis-pinned from a boot-transient probe (Architect, DevEx, catfish). 5. Missing 6. Unbounded Happy to see this land once the observability gap (marker file + a |
…te, driver, egress, timeout) Resolve the six findings from the AI-assisted 5-role adversarial PR review (all confirmed against the diff). Design direction was affirmed; these are implementation hardening. 1. Fail-silent -> observable (finding #1). The install now writes a durable success marker /var/lib/acq/oci-engine-ready (state=configured on a real install; state=base-engine on the functional-engine early-out). The fail-soft paths deliberately do NOT write it, so an operator/CI can tell a provisioned sandbox from one where OCI setup fell soft. scripts/verify checks the marker first. 2. Package-manager flag injection (finding #2). The OCI_ENGINE_PKGS sanitizer allowed a leading '-' token, so "podman --allow-unauthenticated" (apt) / --nogpgcheck (dnf) / --allow-untrusted (apk) word-split into the elevated install and disabled signature verification. Now validated token-by-token, rejecting any leading-'-' or non-word-safe token, failing closed. 3. Unconditional device grant (finding #3). oci-engine-grant-devs.sh now GATES on `command -v podman`: no engine, no chown/chmod of /dev/net/tun + /dev/fuse (avoids needlessly widening egress-relevant device access for a capability that does not exist). 4. Storage driver mis-pinned from a boot-transient probe (finding #4). Driver selection moved to a shared helper (oci-engine-config/oci-engine-storage-driver.sh) sourced by both the create install and the every-boot grant step, converging vfs -> overlay once fuse-overlayfs + /dev/fuse are present. Only rewrites a kit-owned storage.conf (sentinel), never an operator's. 5. Missing caps.network for the registry (finding #5). spec.yaml now declares the Docker Hub registry/auth/blob-CDN hosts the kit's purpose needs, so runtime pulls work even on a strict tier. The distro package mirror stays with the balanced baseline (distro-specific, already in core). 6. Unbounded boot-path self-test (finding #6). The per-boot rootless `podman build` self-test now runs under `timeout` (OCI_ENGINE_SELFTEST_TIMEOUT, default 120s) so a wedged mount cannot hang the boot; a timeout counts as a failed self-test and triggers the vfs fallback. Also folds in the verify base-image clarification (engine-free :shell base; `*-docker` variants run dockerd and would trip the early-out). spec.yaml, README, and docs/decisions updated; validator/sh -n/scan-shell clean; sanitizer, marker, and storage-convergence paths verified with stubbed shells. Co-authored-by: OpenCode Agent <agent@gsa.gov>
|
Thanks for the thorough adversarial pass — all six findings addressed in 1. Fail-silent → observable (marker + verify). The install now writes a durable success marker 2. Package-manager flag injection. Correct catch. The sanitizer permitted a leading 3. Unconditional device grant. 4. Storage driver mis-pinned from a boot-transient probe. Driver selection moved into a shared helper ( 5. Missing 6. Unbounded self-test on the boot path. The per-boot Validation: Live end-to-end run on an engine-free msb base ( |
|
Live end-to-end re-run on hardware (engine-free msb base This exercises the finding #1 marker ( @wz-gsa — all six findings from the adversarial panel are resolved and verified live; could you re-run the review pass when you have a chance? Happy to iterate on anything else it surfaces. |
…012) The docs/decisions note ended with two consecutive blank lines, tripping MD012/no-multiple-blanks in the Markdown Lint CI check. Trim to a single trailing newline. Co-authored-by: OpenCode Agent <agent@gsa.gov>
|
AI-assisted adversarial review (OpenCode), advisory — needs a human to confirm before it drives a change. Reviewed at head I read the six findings from the 09-22 panel and confirmed all six are genuinely fixed at this head — details at the bottom, because a re-review that only reports new problems isn't much use. One new finding, which I'd treat as blocking, and it's specific to a kit whose whole job is running privileged setup. Blocker: root sources a helper from an agent-writable path, on every boot
- path: /home/agent/oci-engine-config/oci-engine-storage-driver.sh
mode: "0644"Two elevated steps then source it:
The sandbox's own threat model makes this reachable: the agent has passwordless sudo, and quickstart's ADR-0030 states the consequence directly — "inside the guest, root-ownership is not a trust boundary against the agent." But this doesn't even need sudo. The file is in the agent's home at mode 0644, owned by the agent. A prompt-injected agent appends one line and waits for the next The create-time call is less interesting (nothing has run yet). The startup call at Worth noting this kit is the reason quickstart #504 built Mechanism 2 —
(2) is the smallest change and matches what this kit already does correctly for its other two staged files. Should-fix: the two elevated scripts are themselves agent-writableSame mechanism, one step removed. This is arguably the kit vocabulary's problem rather than this kit's — every kit that declares a root startup command inherits it, and the sibling kits do the same thing. But this is the kit with the most root surface (two elevated phases, device-permission changes, a package install), so it's where the pattern is worth naming. If option (2) above is taken for the helper, the same argument applies to these two. Nit:
|
Move elevated OCI kit execution through root-owned copies under /usr/local/lib/acq/oci-engine and verify the trusted path ownership. Also revoke stale device grants when podman is absent. Co-authored-by: OpenCode Agent <agent@gsa.gov>
|
AI-assisted follow-up addressing the latest review at Summary of changes since the review:
Verification run locally: Host-side live verification supplied by Bret from @wz-gsa please re-review the root-owned execution path and stale-grant handling when you have a chance. |
|
AI-assisted re-review (OpenCode), advisory — needs a human to confirm before it drives a change. Reviewed current head This re-confirms the second review's own findings, then independently re-verifies the ORIGINAL panel's 6 findings at the current head too — since a root-hardening commit that fixes a new blocker doesn't automatically mean the earlier fixes are still intact, and I wanted to check rather than inherit that on trust. Ran two passes: (1) direct file reads + a live Second review's blocker + 2 nits: confirmed fixedBlocker (root sourcing a helper from an agent-writable path) — fixed correctly. Nit 1 (stale device grant not revoked) — on rereading, this was already handled: the podman-absent branch in Nit 2 (verify doesn't assert the marker) — also already handled: Original panel's 6 findings, re-verified fresh at the current head (not inherited from the changelog)
Recommendation5 of 6 original findings plus the blocker and both nits from the second review are genuinely fixed, verified by direct code reading and a live test — not by trusting the PR's own summary. I'd lean toward approve, with one non-blocking ask: either fail closed (skip the self-test, not run it unbounded) when Given how much this PR has already tightened across three rounds of review, I don't think #6 alone should block merge — flagging it as the one thing I'd want a decision on (fix now vs. tracked follow-up) rather than silently letting it ride. |
Skip the boot-path rootless podman build self-test when timeout is unavailable, and normalize invalid or zero timeout overrides back to the default so operators cannot accidentally disable the bound. Add regression coverage for missing timeout and invalid timeout values. Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
|
AI-assisted update pushed for re-review. Human review required before merge. Summary:
Adversarial review:
Verification transcript:
Please re-review the bounded self-test behavior and timeout normalization when available. |
…o converted CI caught a real defect in the contract test, from a direction I had not anticipated. Rebased onto main (1.10.0) and fixed. WHAT HAPPENED. `oci-engine` (#437) landed on main while this change was in review — an EIGHTH verify script that did not exist when I branched. The test derived its converted set by subtraction: {k: p for k, p in _verify_scripts().items() if k not in UNCONVERTED} so a kit in neither set was treated as CONVERTED by default. The four contract tests then failed against it with "oci-engine/scripts/verify does not source lib/verify-report.sh", which is true but misleading: the real problem was that a kit appeared that nobody had classified. I had guarded both directions I thought of — a kit converted but still listed, and a listed kit that reverted — and missed the one that actually happened. The default was the bug: an unclassified kit silently inherited a contract it does not implement, which is the same shape as the defect this whole change exists to fix (something unmeasured treated as passing). THE FIX. Two explicit sets, CONVERTED and UNCONVERTED, plus `test_every_kit_is_classified` asserting that every verify script appears in exactly one of them. A kit in neither is now a hard error naming the real cause, and the sets cannot name a kit that has no verify script. Proved it by simulating a ninth unclassified kit: AssertionError: verify script(s) in no classification set: ['zz-phantom']. Add each to CONVERTED (if it sources verify-report.sh) or to UNCONVERTED (if it is still on the old two-state verdict). A kit is never converted by default. CORRECTED COUNTS, recounted on the rebased tree rather than carried forward: there are EIGHT verify scripts, not seven, and 35 uncounted `skip`/NOTE sites, not 29 (9 `warn` sites unchanged). oci-engine itself has 303 lines, 2 gates, and 2 `[ "$fail" -eq 0 ]` verdicts, so it has the primary defect too; it is listed as UNCONVERTED for the staged bulk conversion. Stale numbers in the library header and the test docstring are updated. UNCONVERTED is now: agentic-coding-playbook, oci-engine, openchamber, paseo, pi-coding-agent, zscaler-ca-certificate. It must reach empty. Unrelated to the code, but it is why this took a second pass: the CI log could not be read at first because `gh run view --log` failed with `x509: certificate signed by unknown authority`. Root cause is Go-specific and documented in the review workspace, not here — Go's x509.SystemCertPool() omits macOS admin-installed roots (121 of the host's 177), so any Go tool fails against a proxy-intercepted host while `curl` to the same host verifies fine. Reading the log required pointing SSL_CERT_FILE at a bundle including the admin roots. No repo change needed; noted so the next person who sees that error does not mistake it for a CI outage. Verified on the rebased tree: pytest scripts/tests/ -> 420 passed (13 contract tests) unclassified-kit probe -> FAILS with the naming assertion validate_repo.py --root . -> exit 0 validate-kits.py --root . --strict -> all kits OK scan_unsafe_shell.py . -> 0 errors, 0 warnings shellcheck -s sh verify-report.sh -> exit 0 ruff check / ruff format --check -> clean AI assistance disclosed: prepared by an AI agent (OpenCode) on operator instruction. Human review and merge still required.
Summary
AI-assisted. Adds a new neutral acq mixin kit,
oci-engine, that provides a rootless podman OCI container engine as an opt-in capability. This implements the OCI-engine-kit direction from ADR-0030 (in the siblingagentic-coding-quickstartrepo): move the rootless-podman provisioning out of the msb backend adapter and into a neutral, backend-agnostic kit that every acq backend applies through the sharedhybrid/v1vocabulary, off by default.It is consumed opt-in by acq: the quickstart side already added an
ACQ_ENABLE_OCI_KITselection gate that fetches and validates a kit at the pinned patterns ref requiring exactlyschemaVersion: "hybrid/v1",kind: mixin,name: oci-engine. Until this kit merges and is pinned, that opt-in no-ops by design.What it does
podman-compose,fuse-overlayfs,uidmap,passt,slirp4netns) via apt/dnf/apk; selects a storage driver that works on an overlay root (overlay+fuse-overlayfs, elsevfs); writes a Docker-Hub-first registry config (short-name-mode=enforcingby default); shadows the base image's dead-socket docker CLI with adocker→podmanwrapper./dev/net/tun+/dev/fuseto the agent (root:agent,0660) since/devis re-created each boot; and runs a rootless layer-mount self-test that writes a user-levelvfsstorage fallback if the overlay+fuse-overlayfs combo is rejected under rootless.enforcingshort-name mode).Security posture (ADR-0020)
Rootless engine; only install/config and the device grant need root. Exactly two device nodes exposed, group-scoped
0660(not world-writable). Registry resolution is Docker-Hub-first withshort-name-mode=enforcing(least-privilege / prompt-injection defense: ambiguous unqualified names fail closed instead of being silently substituted). No secrets. Egress: none declared — the OS package mirrors are already in the balanced baselinecoretier (isolation ADR-0002); a strict tier fails the install soft.Verification
make validate-kits(validate-kits.py --strict): All 8 acq-kits valid, no errors/warnings.sh -nclean on all 4/bin/shscripts;bash -nclean on the verify script; ShellCheck (severity=warning) clean.sbx create).scripts/verifyis a host-side live check (needssbx; skipped in CI, like the zscaler kit's verify) — not run here (no sandbox-capable host).Review
This kit was written and then put through an independent adversarial review before this PR. That review found and this branch fixed: (1) a blocker where
set -eu+ bare package-manager calls defeated the documented fail-soft and would have failedsbx create; (2) a dropped rootlessvfsrecovery step vs. the reference implementation; (3) a verify script that could pass green on a rootful engine. All three are fixed in commit3d9010b, with adocs/decisions/note recording the port and the fail-soft + vfs-fallback choices.Rollback
Revert this PR. The kit is additive and off by default; nothing selects it until the quickstart side pins this ref and an operator sets
ACQ_ENABLE_OCI_KIT. The msb adapter's existing podman provisioning remains the active OCI mechanism in the meantime.Related